{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Q-Network Variant NoisyNet DQN implementation (PyTorch).\n",
    "\n",
    "In this notebook, we will implement NoisyNet DQN variant of DQN. First part of the notebook is all about the vanilla DQN and how we train - a copy of `6_1_dqn_pytorch.ipynb`. The NoisyNet DQN variant is after these sections on vanilla DQN. \n",
    "\n",
    "\n",
    "### RECAP\n",
    "\n",
    "$$ \n",
    "\\DeclareMathOperator*{\\max}{max}$$\n",
    "\n",
    "Q Learning control is carried out by sampling step by step and updating Q values at each step. We use ε-greedy policy to explore and generate samples. However, the policy learnt is a deterministic greedy policy with no exploration. We can carryout updates online i.e. we take a step and use `(current state, action, reward and next_state)` tuple to update. \n",
    "\n",
    "In case of function approximation using neural network, the input to the network is the state and output is the q(s,a) for all the actions in the state `s`. It is denoted as $ \\hat{q}(s_t,a; w_{t}) $, where $w_{t}$ is the weights of the neural network which we are trying to learn as part of DQN learning. \n",
    "\n",
    "We use two networks, one target network to get the max q-value of next state denoted by $ \\max_a \\hat {q}(s_{t+1},a; w^{-}_{t}) $ and the primary network with weights $w_{t}$ which we are updated based on back propagation of the TD error through the network.\n",
    "\n",
    "The Update equation is given below. This is the online version:\n",
    "$$ w_{t+1} \\leftarrow w_t + \\alpha [ R_{t+1} + \\gamma . \\max_{a} \\hat{q}(S_{t+1},a,w^{-}_{t}) – \\hat{q}(S_t,A_t,w_t)] \\nabla \\hat{q}(S_t,A_t,w_t)$$\n",
    "\n",
    "Online update with neural network with millions of weights does not work well. Accordingly, We use experience replay (aka Replay Buffer).  We use a behavior policy to explore the environment and store the samples `(s, a, r, s', done)` in a buffer. The samples are generated using an exploratory behavior policy while we improve a deterministic target policy using q-values. \n",
    "\n",
    "Therefore, we can always use older samples from behavior policy and apply them again and again. We can keep the buffer size fixed to some pre-determined size and keep deleting the older samples as we collect new ones. This process makes learning sample efficient by reusing a sample multiple times and also removing temporal dependence of the samples we would otherwise see while following a trajectory.\n",
    "\n",
    "The update equation with batch update with minor modifications is given below. We collect samples of transitions (current state, action, reward, next state) in a buffer. Where each sample is denoted as a tuple: \n",
    "\n",
    "$$ (s_{i}, a_{i}, r_{i}, s^{'}_{i}, done_{i})$$\n",
    "\n",
    "Subscript (i) denotes ith sample. We take N samples from experience replay selecting randomly and update the weights. Subscript (t) denotes the index of weight updates. If the current state is done, as denoted by `done` flag, the target is just the reward as terminal states have zero value. The final update equation is as given below:\n",
    "\n",
    "$$w_{t+1} \\leftarrow w_t + \\alpha \\frac{1}{N} \\sum_{i=1}^{N} \\left[ r_i + \\left( (1-done_i) . \\gamma .  \\max_{a^{'}} \\hat{q}(s_{i}^{'},a^{'};w^{-}_{t}) \\right) – \\hat{q}(s_i,a_i;w_t) \\right] \\nabla \\hat{q}(s_i,a_i;w_t)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.optim as optim\n",
    "\n",
    "import gym\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.signal import convolve, gaussian\n",
    "import math\n",
    "import os\n",
    "import io\n",
    "import base64\n",
    "import time\n",
    "from tqdm import trange\n",
    "import glob\n",
    "import random\n",
    "from collections import namedtuple\n",
    "\n",
    "from IPython.display import HTML, clear_output\n",
    "\n",
    "%matplotlib inline\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Environment - CartPole \n",
    "\n",
    "We can use the setup here to run on any environment which has state as a single vector and actions are discrete. We will build it on Cart Pole and they try to run this on many other environments like Lunar Lander and others.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_env(env_name, seed=None):\n",
    "    # remove time limit wrapper from environment\n",
    "    env = gym.make(env_name).unwrapped\n",
    "    if seed is not None:\n",
    "        env.seed(seed)\n",
    "    return env"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAATO0lEQVR4nO3dXaxd5Z3f8e+PE+MwhplAfCCOX7AbPBUQdUx05ImUmZZJ6OAy1ZhcpDJSI18gORdESlSUFmakTnJhaVpNXnqRRCIFjZVkQhwlBCdK2wGaKI2UYkzGgI1x8AQDBzu2eTEGJjZ++ffiLMoGH9vb5yXHz9nfj7S11/6vZ+39f5D9Y53nrO2VqkKS1I7zZroBSdLZMbglqTEGtyQ1xuCWpMYY3JLUGINbkhozbcGdZFWSnUl2Jbltuj5HkgZNpuM67iRDwC+Bfw2MAg8BN1XV41P+YZI0YKbrjHslsKuqflVVrwN3A6un6bMkaaC8Y5redyHwbM/rUeAPTzV4/vz5tXTp0mlqRZLas3v3bp5//vmMt2+6gnu8D3vLmkySdcA6gCVLlrBly5ZpakWS2jMyMnLKfdO1VDIKLO55vQjY0zugqu6oqpGqGhkeHp6mNiRp9pmu4H4IWJ5kWZLzgTXApmn6LEkaKNOyVFJVx5J8EvhfwBBwV1Vtn47PkqRBM11r3FTVj4AfTdf7S9Kg8puTktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaM6lblyXZDbwCHAeOVdVIkkuAbwNLgd3Av6uqlybXpiTpDVNxxv0nVbWiqka617cBD1TVcuCB7rUkaYpMx1LJamBDt70BuHEaPkOSBtZkg7uAv0/ycJJ1Xe2yqtoL0D1fOsnPkCT1mNQaN/ChqtqT5FLgviRP9HtgF/TrAJYsWTLJNiRpcEzqjLuq9nTP+4F7gJXAviQLALrn/ac49o6qGqmqkeHh4cm0IUkDZcLBnWRekove2Ab+FNgGbALWdsPWAvdOtklJ0psms1RyGXBPkjfe5++q6n8meQjYmORm4BngY5NvU5L0hgkHd1X9CviDceovAB+ZTFOSpFPzm5OS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSY84Y3EnuSrI/ybae2iVJ7kvyZPd8cc++25PsSrIzyfXT1bgkDap+zrj/Flj1ttptwANVtRx4oHtNkquANcDV3TFfSTI0Zd1Kks4c3FX1U+DFt5VXAxu67Q3AjT31u6vqSFU9BewCVk5Nq5IkmPga92VVtRege760qy8Enu0ZN9rVTpJkXZItSbYcOHBggm1I0uCZ6l9OZpxajTewqu6oqpGqGhkeHp7iNiRp9ppocO9LsgCge97f1UeBxT3jFgF7Jt6eJOntJhrcm4C13fZa4N6e+pokc5MsA5YDmyfXoiSp1zvONCDJt4BrgflJRoG/Av4a2JjkZuAZ4GMAVbU9yUbgceAYcEtVHZ+m3iVpIJ0xuKvqplPs+sgpxq8H1k+mKUnSqfnNSUlqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTljcCe5K8n+JNt6ap9N8lySrd3jhp59tyfZlWRnkuunq3FJGlT9nHH/LbBqnPoXq2pF9/gRQJKrgDXA1d0xX0kyNFXNSpL6CO6q+inwYp/vtxq4u6qOVNVTwC5g5ST6kyS9zWTWuD+Z5NFuKeXirrYQeLZnzGhXO0mSdUm2JNly4MCBSbQhSYNlosH9VeB9wApgL/D5rp5xxtZ4b1BVd1TVSFWNDA8PT7ANSRo8EwruqtpXVcer6gTwNd5cDhkFFvcMXQTsmVyLkqReEwruJAt6Xn4UeOOKk03AmiRzkywDlgObJ9eiJKnXO840IMm3gGuB+UlGgb8Crk2ygrFlkN3AJwCqanuSjcDjwDHglqo6Pi2dS9KAOmNwV9VN45TvPM349cD6yTQlSTo1vzkpSY0xuCWpMQa3JDXG4JakxhjcktQYg1sD6/XXDnLouSc4duSfZroV6ayc8XJAabb4zUt7Gf35xv//+vXXXuLwS3v553/+GS58zxUz2Jl0dgxuDYzjr/8Th0Yfn+k2pElzqUSSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3BoYc37n95j7uyff3/SVvU9SNe6tUaVzksGtgTH3ovlccMnCk+ovP/PoDHQjTZzBLUmNOWNwJ1mc5MdJdiTZnuRTXf2SJPclebJ7vrjnmNuT7EqyM8n10zkBSRo0/ZxxHwNuraorgQ8CtyS5CrgNeKCqlgMPdK/p9q0BrgZWAV9JMjQdzUvSIDpjcFfV3qr6Rbf9CrADWAisBjZ0wzYAN3bbq4G7q+pIVT0F7AJWTnHfkjSwzmqNO8lS4BrgQeCyqtoLY+EOXNoNWwg823PYaFd7+3utS7IlyZYDBw5MoHVJGkx9B3eSC4HvAp+uqkOnGzpO7aRrrarqjqoaqaqR4eGTL9GSJI2vr+BOMoex0P5mVX2vK+9LsqDbvwDY39VHgcU9hy8C9kxNu5Kkfq4qCXAnsKOqvtCzaxOwttteC9zbU1+TZG6SZcByYPPUtSxJg62fO+B8CPg48FiSrV3tL4C/BjYmuRl4BvgYQFVtT7IReJyxK1JuqarjU924JA2qMwZ3Vf2M8detAT5yimPWA+sn0Zck6RT85qQGyoXvWX5S7ehrBzlyyCub1A6DWwPl95a8H/LWHyBff/VFDr/k78/VDoNbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbg2UobnzuOCSRSfVX9nzS6pOut+HdE4yuDVQ5lxwEfMuXXpS/eVnt4HBrUYY3JLUGINbkhpjcEtSYwxuSWpMPzcLXpzkx0l2JNme5FNd/bNJnkuytXvc0HPM7Ul2JdmZ5PrpnIAkDZp+bhZ8DLi1qn6R5CLg4ST3dfu+WFV/0zs4yVXAGuBq4L3A/Ul+3xsGS9LUOOMZd1XtrapfdNuvADuAhac5ZDVwd1UdqaqngF3AyqloVpJ0lmvcSZYC1wAPdqVPJnk0yV1JLu5qC4Fnew4b5fRBL0k6C30Hd5ILge8Cn66qQ8BXgfcBK4C9wOffGDrO4Sd9syHJuiRbkmw5cMA7bEtSv/oK7iRzGAvtb1bV9wCqal9VHa+qE8DXeHM5ZBRY3HP4IuCkW2hX1R1VNVJVI8PDw5OZgyQNlH6uKglwJ7Cjqr7QU1/QM+yjwLZuexOwJsncJMuA5cDmqWtZmpwL37OcnDf0ltrxI6/xmxdHZ6gj6ez0c1XJh4CPA48l2drV/gK4KckKxpZBdgOfAKiq7Uk2Ao8zdkXKLV5RonPJ7y68kpw3RJ1484/lscOv8tqBp/md+UtmsDOpP2cM7qr6GeOvW//oNMesB9ZPoi9J0in4zUlJaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwa2Bc96c85l32ftOqr+695ecOH5sBjqSzo7BrYEzNOedXPieK06qH3ruibf8+yXSucrglqTGGNyS1Jh+/llXqQn3338/X/7yl/sa+8dXzONfLZ/3ltrBgy9x0003cfT4STdsOsnixYv50pe+xHnnee6j3z6DW7PG008/zfe///2+xg7/2Qf4oyv+kGMnzgcgOcHhwy/xgx/8gMOvn/kXlFdeeSVVZw54aToY3BpIJziPx17+Y/YeXgbAnBzh8vM2zXBXUn/8OU8D6eWj8/n14aUcrzkcrzkcPnEhW1/+E46X5zI69xncGkgHjizmWJ3/ltqxE3NmqBvp7PRzs+B3Jtmc5JEk25N8rqtfkuS+JE92zxf3HHN7kl1Jdia5fjonIE3Eey/YxZwcfkvtgqFXCa5b69zXzxn3EeDDVfUHwApgVZIPArcBD1TVcuCB7jVJrgLWAFcDq4CvJBka742lmTJv6BDL5m1j3tBBXnn5WQ69uIPho/cQ/Oakzn393Cy4gFe7l3O6RwGrgWu7+gbgJ8B/6up3V9UR4Kkku4CVwM9P9RlHjx7l17/+9cRmIHUOHTrU99ifPfY0Lxz6bxTh/zz6NC8e+g2hONHnlSLHjx9n3759Xg6oaXP06NFT7uvrNzHdGfPDwBXAl6vqwSSXVdVegKram+TSbvhC4P/2HD7a1U7phRde4Otf/3o/rUin9NBDD/U99olnnueJZ55/S+1sFkkOHjzIN77xDZKcxVFS/1544YVT7usruKvqOLAiybuAe5K8/zTDx/uTfNLfiSTrgHUAS5Ys4TOf+Uw/rUindOedd/Kd73znt/JZ7373u7n11lsZGnIVUNPj29/+9in3ndXPeVV1kLElkVXAviQLALrn/d2wUWBxz2GLgD3jvNcdVTVSVSPDw8Nn04YkDbR+rioZ7s60SXIBcB3wBLAJWNsNWwvc221vAtYkmZtkGbAc2DzFfUvSwOpnqWQBsKFb5z4P2FhVP0zyc2BjkpuBZ4CPAVTV9iQbgceBY8At3VKLJGkK9HNVyaPANePUXwA+copj1gPrJ92dJOkkXsskSY0xuCWpMf6LOpo1Lr/8cm688cbfymctXrzYa7g1YwxuzRrXXXcd11133Uy3IU07l0okqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmP6uVnwO5NsTvJIku1JPtfVP5vkuSRbu8cNPcfcnmRXkp1Jrp/OCUjSoOnn3+M+Any4ql5NMgf4WZL/0e37YlX9Te/gJFcBa4CrgfcC9yf5fW8YLElT44xn3DXm1e7lnO5RpzlkNXB3VR2pqqeAXcDKSXcqSQL6XONOMpRkK7AfuK+qHux2fTLJo0nuSnJxV1sIPNtz+GhXkyRNgb6Cu6qOV9UKYBGwMsn7ga8C7wNWAHuBz3fDx7sR30ln6EnWJdmSZMuBAwcm0LokDaazuqqkqg4CPwFWVdW+LtBPAF/jzeWQUWBxz2GLgD3jvNcdVTVSVSPDw8MT6V2SBlI/V5UMJ3lXt30BcB3wRJIFPcM+CmzrtjcBa5LMTbIMWA5sntKuJWmA9XNVyQJgQ5IhxoJ+Y1X9MMnXk6xgbBlkN/AJgKranmQj8DhwDLjFK0okaeqcMbir6lHgmnHqHz/NMeuB9ZNrTZI0Hr85KUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGpOqmukeSHIAeA14fqZ7mQbzcV6tma1zc15tubyqhsfbcU4EN0CSLVU1MtN9TDXn1Z7ZOjfnNXu4VCJJjTG4Jakx51Jw3zHTDUwT59We2To35zVLnDNr3JKk/pxLZ9ySpD7MeHAnWZVkZ5JdSW6b6X7OVpK7kuxPsq2ndkmS+5I82T1f3LPv9m6uO5NcPzNdn1mSxUl+nGRHku1JPtXVm55bkncm2ZzkkW5en+vqTc/rDUmGkvxDkh92r2fLvHYneSzJ1iRbutqsmNuEVNWMPYAh4B+BfwacDzwCXDWTPU1gDv8S+ACwraf2X4Hbuu3bgP/SbV/VzXEusKyb+9BMz+EU81oAfKDbvgj4Zdd/03MDAlzYbc8BHgQ+2Pq8eub3H4C/A344W/4sdv3uBua/rTYr5jaRx0yfca8EdlXVr6rqdeBuYPUM93RWquqnwItvK68GNnTbG4Abe+p3V9WRqnoK2MXYf4NzTlXtrapfdNuvADuAhTQ+txrzavdyTvcoGp8XQJJFwJ8B/72n3Py8TmM2z+20Zjq4FwLP9rwe7Wqtu6yq9sJYAAKXdvUm55tkKXANY2enzc+tW07YCuwH7quqWTEv4EvAfwRO9NRmw7xg7H+uf5/k4STrutpsmdtZe8cMf37Gqc3my1yam2+SC4HvAp+uqkPJeFMYGzpO7ZycW1UdB1YkeRdwT5L3n2Z4E/NK8m+B/VX1cJJr+zlknNo5N68eH6qqPUkuBe5L8sRpxrY2t7M202fco8DinteLgD0z1MtU2pdkAUD3vL+rNzXfJHMYC+1vVtX3uvKsmBtAVR0EfgKsov15fQj48yS7GVty/HCSb9D+vACoqj3d837gHsaWPmbF3CZipoP7IWB5kmVJzgfWAJtmuKepsAlY222vBe7tqa9JMjfJMmA5sHkG+jujjJ1a3wnsqKov9Oxqem5JhrszbZJcAFwHPEHj86qq26tqUVUtZezv0f+uqn9P4/MCSDIvyUVvbAN/CmxjFsxtwmb6t6PADYxdsfCPwF/OdD8T6P9bwF7gKGP/p78ZeDfwAPBk93xJz/i/7Oa6E/g3M93/aeb1R4z9ePkosLV73ND63IB/AfxDN69twH/u6k3P621zvJY3ryppfl6MXXX2SPfY/kZOzIa5TfThNyclqTEzvVQiSTpLBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY35f7TbvVd1yFXnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ENV_NAME = 'CartPole-v1'\n",
    "\n",
    "env = make_env(ENV_NAME)\n",
    "env.reset()\n",
    "plt.imshow(env.render(\"rgb_array\"))\n",
    "state_shape, n_actions = env.observation_space.shape, env.action_space.n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building a network using pytorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us build a DQN agent using a simple network. We now need to build a neural network that can map states/observations to state q-values. The observation space and action space is as given below for CartPole\n",
    "\n",
    "    Observation:\n",
    "        Type: Box(4)\n",
    "        Num     Observation               Min                     Max\n",
    "        0       Cart Position             -4.8                    4.8\n",
    "        1       Cart Velocity             -Inf                    Inf\n",
    "        2       Pole Angle                -0.418 rad (-24 deg)    0.418 rad (24 deg)\n",
    "        3       Pole Angular Velocity     -Inf                    Inf\n",
    "    Actions:\n",
    "        Type: Discrete(2)\n",
    "        Num   Action\n",
    "        0     Push cart to the left\n",
    "        1     Push cart to the right\n",
    "        \n",
    "\n",
    "The model will be a simple one with 2 hidden payers with Relu activation and final layer being logits with dimension equation to number of actions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cpu')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DQNAgent(nn.Module):\n",
    "    def __init__(self, state_shape, n_actions, epsilon=0):\n",
    "\n",
    "        super().__init__()\n",
    "        self.epsilon = epsilon\n",
    "        self.n_actions = n_actions\n",
    "        self.state_shape = state_shape\n",
    "        \n",
    "        state_dim = state_shape[0]\n",
    "        # a simple NN with state_dim as input vector (inout is state s)\n",
    "        # and self.n_actions as output vector of logits of q(s, a)\n",
    "        self.fc1 = nn.Linear(state_dim, 64)\n",
    "        self.fc2 = nn.Linear(64, 128)\n",
    "        self.fc3 = nn.Linear(128, 32)\n",
    "        self.q = nn.Linear(32, n_actions)\n",
    "        \n",
    "    def forward(self, state_t):\n",
    "        # pass the state at time t through the newrok to get Q(s,a)\n",
    "        x = F.relu(self.fc1(state_t))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = F.relu(self.fc3(x))\n",
    "        qvalues = self.q(x)\n",
    "        return qvalues\n",
    "\n",
    "    def get_qvalues(self, states):\n",
    "        # input is an array of states in numpy and outout is Qvals as numpy array\n",
    "        states = torch.tensor(states, device=device, dtype=torch.float32)\n",
    "        qvalues = self.forward(states)\n",
    "        return qvalues.data.cpu().numpy()\n",
    "\n",
    "    def sample_actions(self, qvalues):\n",
    "        # sample actions from a batch of q_values using epsilon greedy policy\n",
    "        epsilon = self.epsilon\n",
    "        batch_size, n_actions = qvalues.shape\n",
    "        random_actions = np.random.choice(n_actions, size=batch_size)\n",
    "        best_actions = qvalues.argmax(axis=-1)\n",
    "        should_explore = np.random.choice(\n",
    "            [0, 1], batch_size, p=[1-epsilon, epsilon])\n",
    "        return np.where(should_explore, random_actions, best_actions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = DQNAgent(state_shape, n_actions, epsilon=0.5).to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(env, agent, n_games=1, greedy=False, t_max=10000):\n",
    "    rewards = []\n",
    "    for _ in range(n_games):\n",
    "        s = env.reset()\n",
    "        reward = 0\n",
    "        for _ in range(t_max):\n",
    "            qvalues = agent.get_qvalues([s])\n",
    "            action = qvalues.argmax(axis=-1)[0] if greedy else agent.sample_actions(qvalues)[0]\n",
    "            s, r, done, _ = env.step(action)\n",
    "            reward += r\n",
    "            if done:\n",
    "                break\n",
    "\n",
    "        rewards.append(reward)\n",
    "    return np.mean(rewards)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11.0\n"
     ]
    }
   ],
   "source": [
    "print(evaluate(env, agent, n_games=1))\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experience replay\n",
    "\n",
    "We will use the replay buffer we saw in chapter 4 listings. Replay buffer is very important in DQN to break the correlation between samples. We use a behavior policy (epsilon greedy) to sample from the environment and store the transitions (s,a,r,s',done) into a buffer. These samples are used multiple times in a learning making the process sample efficient. \n",
    "\n",
    "The interface to ReplayBuffer is:\n",
    "* `exp_replay.add(state, action, reward, next_state, done)` - saves (s,a,r,s',done) tuple into the buffer\n",
    "* `exp_replay.sample(batch_size)` - returns states, actions, rewards, next_states and done_flags for `batch_size` random samples.\n",
    "* `len(exp_replay)` - returns number of elements stored in replay buffer.\n",
    "\n",
    "We have modified the implementation a bit to make it more efficient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ReplayBuffer:\n",
    "    def __init__(self, size):\n",
    "        self.size = size #max number of items in buffer\n",
    "        self.buffer =[] #array to holde buffer\n",
    "        self.next_id = 0\n",
    "    \n",
    "    def __len__(self):\n",
    "        return len(self.buffer)\n",
    "    \n",
    "    def add(self, state, action, reward, next_state, done):\n",
    "        item = (state, action, reward, next_state, done)\n",
    "        if len(self.buffer) < self.size:\n",
    "           self.buffer.append(item)\n",
    "        else:\n",
    "            self.buffer[self.next_id] = item\n",
    "        self.next_id = (self.next_id + 1) % self.size\n",
    "        \n",
    "    def sample(self, batch_size):\n",
    "        idxs = np.random.choice(len(self.buffer), batch_size)\n",
    "        samples = [self.buffer[i] for i in idxs]\n",
    "        states, actions, rewards, next_states, done_flags = list(zip(*samples))\n",
    "        return np.array(states), np.array(actions), np.array(rewards), np.array(next_states), np.array(done_flags)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def play_and_record(start_state, agent, env, exp_replay, n_steps=1):\n",
    "\n",
    "    s = start_state\n",
    "    sum_rewards = 0\n",
    "\n",
    "    # Play the game for n_steps and record transitions in buffer\n",
    "    for _ in range(n_steps):\n",
    "        qvalues = agent.get_qvalues([s])\n",
    "        a = agent.sample_actions(qvalues)[0]        \n",
    "        next_s, r, done, _ = env.step(a)\n",
    "        sum_rewards += r\n",
    "        exp_replay.add(s, a, r, next_s, done)\n",
    "        if done:\n",
    "            s = env.reset()\n",
    "        else:\n",
    "            s = next_s\n",
    "        \n",
    "    return sum_rewards, s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Target network\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<All keys matched successfully>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "target_network = DQNAgent(agent.state_shape, agent.n_actions, epsilon=0.5).to(device)\n",
    "target_network.load_state_dict(agent.state_dict())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning with... Q-learning\n",
    "Here we write a function similar to tabular q-learning. We will calculate average TD error per batch using the equation: \n",
    "\n",
    "$$ L =  \\frac{1}{N} \\sum_{i=1}^{N} \\left[ r_i + \\left( (1-done_i) . \\gamma .  \\max_{a^{'}} \\hat{q}(s_{i}^{'},a^{'};w^{-}_{t}) \\right) – \\hat{q}(s_i,a_i;w_t) \\right]^2$$\n",
    "\n",
    "\n",
    "$$ \\nabla_w L =   - \\frac{1}{N} \\sum_{i=1}^{N} \\left[ r_i + \\left( (1-done_i) . \\gamma .  \\max_{a^{'}} \\hat{q}(s_{i}^{'},a^{'};w^{-}_{t}) \\right) – \\hat{q}(s_i,a_i;w_t) \\right] \\nabla \\hat{q}(s_i,a_i;w_t)$$\n",
    "\n",
    "\n",
    "$\\hat{q}(s',A;w^{-})$ is calculated using target network whose weights are held constant and refreshed periodically from the agent learning network. \n",
    "\n",
    "Target is given by following:\n",
    "* non terminal state: $r_i +  \\gamma .  \\max_{a^{'}} \\hat{q}(s_{i}^{'},a^{'};w^{-}_{t})$\n",
    "* terminal state: $ r_i $\n",
    "\n",
    "We then carryout back propagation through the agent network to update the weights using equation below:\n",
    "\n",
    "\n",
    "$$ \n",
    "\\DeclareMathOperator*{\\max}{max} w_{t+1} \\leftarrow w_t - \\alpha \\nabla_{w}L$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def td_loss_dqn(agent, target_network, states, actions, rewards, next_states, done_flags,\n",
    "                    gamma=0.99, device=device):\n",
    "\n",
    "    # convert numpy array to torch tensors\n",
    "    states = torch.tensor(states, device=device, dtype=torch.float)\n",
    "    actions = torch.tensor(actions, device=device, dtype=torch.long)\n",
    "    rewards = torch.tensor(rewards, device=device, dtype=torch.float)\n",
    "    next_states = torch.tensor(next_states, device=device, dtype=torch.float)\n",
    "    done_flags = torch.tensor(done_flags.astype('float32'),device=device,dtype=torch.float)\n",
    "\n",
    "    # get q-values for all actions in current states\n",
    "    # use agent network\n",
    "    q_s = agent(states)\n",
    "\n",
    "    # select q-values for chosen actions\n",
    "    q_s_a = q_s[range(\n",
    "        len(actions)), actions]\n",
    "\n",
    "    with torch.no_grad():\n",
    "        # compute q-values for all actions in next states\n",
    "        # use target network\n",
    "        q_s1 = target_network(next_states)\n",
    "\n",
    "\n",
    "        # compute Qmax(next_states, actions) using predicted next q-values\n",
    "        q_s1_a1max,_ = torch.max(q_s1, dim=1)\n",
    "\n",
    "        # compute \"target q-values\" \n",
    "        target_q = rewards + gamma * q_s1_a1max * (1-done_flags)\n",
    "\n",
    "    # mean squared error loss to minimize\n",
    "    loss = torch.mean((q_s_a - target_q.detach()) ** 2)\n",
    "\n",
    "    return loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def epsilon_schedule(start_eps, end_eps, step, final_step):\n",
    "    return start_eps + (end_eps-start_eps)*min(step, final_step)/final_step\n",
    "\n",
    "def smoothen(values):\n",
    "    kernel = gaussian(100, std=100)\n",
    "    kernel = kernel / np.sum(kernel)\n",
    "    return convolve(values, kernel, 'valid')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Main loop\n",
    "\n",
    "We now carryout the training on DQN setup above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_agent(env, agent, target_network, optimizer, td_loss_fn):\n",
    "    \n",
    "    state = env.reset()\n",
    "    # let us fill experience replay with some samples using full random policy\n",
    "    exp_replay = ReplayBuffer(10**4)\n",
    "    for i in range(100):\n",
    "        play_and_record(state, agent, env, exp_replay, n_steps=10**2)\n",
    "        if len(exp_replay) == 10**4:\n",
    "            break\n",
    "    print('Finished filling buffer with: {} samples'.format(len(exp_replay)))\n",
    "\n",
    "    mean_rw_history = []\n",
    "    td_loss_history = []\n",
    "    state = env.reset()\n",
    "    for step in trange(total_steps + 1):\n",
    "\n",
    "        # reduce exploration as we progress\n",
    "        agent.epsilon = epsilon_schedule(start_epsilon, end_epsilon, step, eps_decay_final_step)\n",
    "\n",
    "        # take timesteps_per_epoch and update experience replay buffer\n",
    "        _, state = play_and_record(state, agent, env, exp_replay, timesteps_per_epoch)\n",
    "\n",
    "        # train by sampling batch_size of data from experience replay\n",
    "        states, actions, rewards, next_states, done_flags = exp_replay.sample(batch_size)\n",
    "\n",
    "\n",
    "        # loss = <compute TD loss>\n",
    "        optimizer.zero_grad()\n",
    "        loss = td_loss_fn(agent, target_network, \n",
    "                          states, actions, rewards, next_states, done_flags,                  \n",
    "                          gamma=0.99,\n",
    "                          device=device)\n",
    "\n",
    "        loss.backward()\n",
    "        grad_norm = nn.utils.clip_grad_norm_(agent.parameters(), max_grad_norm)\n",
    "        optimizer.step()\n",
    "\n",
    "        if step % loss_freq == 0:\n",
    "            td_loss_history.append(loss.data.cpu().item())\n",
    "\n",
    "        if step % refresh_target_network_freq == 0:\n",
    "            # Load agent weights into target_network\n",
    "            target_network.load_state_dict(agent.state_dict())\n",
    "\n",
    "        if step % eval_freq == 0:\n",
    "            # eval the agent\n",
    "            mean_rw_history.append(evaluate(\n",
    "                make_env(ENV_NAME, seed=step), agent, n_games=3, greedy=True, t_max=1000)\n",
    "            )\n",
    "\n",
    "            clear_output(True)\n",
    "            print(\"buffer size = %i, epsilon = %.5f\" %\n",
    "                  (len(exp_replay), agent.epsilon))\n",
    "\n",
    "            plt.figure(figsize=[16, 5])\n",
    "            plt.subplot(1, 2, 1)\n",
    "            plt.title(\"Mean reward per episode\")\n",
    "            plt.plot(mean_rw_history)\n",
    "            plt.grid()\n",
    "\n",
    "            assert not np.isnan(td_loss_history[-1])\n",
    "            plt.subplot(1, 2, 2)\n",
    "            plt.title(\"TD loss history (smoothened)\")\n",
    "            plt.plot(smoothen(td_loss_history))\n",
    "            plt.grid()\n",
    "\n",
    "            plt.show()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#setup some parameters for training\n",
    "timesteps_per_epoch = 1\n",
    "batch_size = 32\n",
    "total_steps = 3 * 10**4\n",
    "\n",
    "# set exploration epsilon \n",
    "start_epsilon = 1\n",
    "end_epsilon = 0.05\n",
    "eps_decay_final_step = 2 * 10**4\n",
    "\n",
    "# setup spme frequency for loggind and updating target network\n",
    "loss_freq = 50\n",
    "refresh_target_network_freq = 100\n",
    "eval_freq = 1000\n",
    "\n",
    "# to clip the gradients\n",
    "max_grad_norm = 5000\n",
    "\n",
    "#set up random numbers\n",
    "seed=0\n",
    "random.seed(seed)\n",
    "np.random.seed(seed)\n",
    "torch.manual_seed(seed) \n",
    "\n",
    "#init environment\n",
    "ENV_NAME = 'CartPole-v1'\n",
    "env = make_env(ENV_NAME, seed)\n",
    "state_dim = env.observation_space.shape\n",
    "n_actions = env.action_space.n\n",
    "\n",
    "#init agent, target network and Optimizer\n",
    "agent = DQNAgent(state_dim, n_actions, epsilon=1).to(device)\n",
    "target_network = DQNAgent(state_dim, n_actions, epsilon=1).to(device)\n",
    "target_network.load_state_dict(agent.state_dict())\n",
    "optimizer = torch.optim.Adam(agent.parameters(), lr=1e-4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "buffer size = 10000, epsilon = 0.05000\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6UAAAE/CAYAAAC+Q2VKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAACAhUlEQVR4nO3dd3hcV7X38e9W792SZcuW3HuJ7dhOcaL0kE5ooYYQCIRy4QIXCOXCBcIbLlx6KAECAVKBhIR024kcO8XdibvlJlmWLFm9ayTNfv+YkSLbkjWSZnRGM7/P8/iR5szMmXX2yHNmnb332sZai4iIiIiIiIgTIpwOQERERERERMKXklIRERERERFxjJJSERERERERcYySUhEREREREXGMklIRERERERFxjJJSERERERERcYySUpExxBhTZIz5uNNxDJUx5rfGmG/5eZ8fNcZs8Oc+RUTEN6P5GWyMKTDGWGNM1AD3f90Y84fRiGWojDGfNMb8zOk4ziZYvlsYY75jjPmb9/ccY8xeY0ys03HJ6FBSKo4zxhw1xriMMVmnbd/hPQkVOBSa+Im19lPW2u85HYeISDgyxjT3+ec2xrT1uf1BbzLQaYxp8v47YIz5lTEm1+nYfWGt/YG1dtCkarSTL2NMDPBN4Eej9ZqD6Zv4BTNrbSXwMnCH07HI6FBSKsHiCPD+nhvGmAVAvHPhvG2gK7MBfk1jjHHs/6cTxywiIoFhrU3q+QeUAtf32fag92GPWmuTgQzgncB4YOtYSUwDbZjn5RuBfdba44GIKQw8CHzS6SBkdCgplWDxV+AjfW7fCvyl7wOMMbHGmB8bY0qNMZXeIaHx3vvSjTFPG2NOGmPqvL/n9XlukTHme8aYV71XgV88vWe2z2MLjTFlxpivGmNOAH8yxkQYY75mjDlkjKkxxjxmjMnwPv4BY8yXvL9P9Pbuftp7e7oxptZ7MvMlxruNMa8CrcBUY8wVxph9xpgGY8yvADNQA3qvfv7DGPOo9xi3GWMW9bl/gjHmn97XP2KM+Y9+nvs3Y0wj8NF+9n+29u9ps68bY6q9vd8f7PPcPxtjvu/9Pct77PXetlnfc6I3xszxtkO9MWa3MeaGPvvINMY8ZYxpNMZsAqadFt9sY8xq7z73G2PeO1BbiYhI/6y1ndba3cD7gJPAl3x5njHmfGPMZu/5arMx5vw+933UGHPYe2460nN+8J4j13mfU22MeXSQl/mg9xxUbYz5Rp/99x32Gec9l9V4zyWbjWco6N3AKuBXxtND/Csf4j79vPwlY8zW0477S8aYfw0Q7zuAdX0e229sfV7r+8aY17zx/dt73nvQe97bbPqMHBsk7gne82WtMeagMeYT3u1XA18H3ud9jTf7xJpvBviOZIxZ6Y2r3hjzpjGm8LQ2GvD71SDPneJ9/5uMMauB07+XbcTzXSh/gPaVEKKkVILFG0CKNymJxHMyPH14yQ+BmcBiYDowEfhv730RwJ+AfGAy0Ab86rTnfwC4DcgGYoAvnyWe8XiuFufjGTryH8BNwMXABKAOuNf72HVAoff3i4HD3p8AFwHrrbXWxxg/7H29ZKAB+CeeoT9ZwCHggrPEDJ6rsn/3xv4Q8C9jTLQ36fs38CaedrsM+IIx5qrTnvsPIA3P1cnTna39wdNmWd7ttwL3GWNm9bOfLwFlwDggB88J0hpjor0xvojnPfoc8GCffdwLtAO5wMe8/wAwxiQCq73HnI2n1/3Xxph5A7aUiIgMyFrbDTyJJ5E7K+O5SPsM8AsgE/gJ8Iw3qUr0bn+Htyf2fGCH96nfw/OZnw7kAb8c5KUuBGbhOYf9tzFmTj+PuRVIBSZ5Y/kU0Gat/QawHvist4f4s2eLu8/++p6XfwFMOe11P4Tnwnp/FgD7B4utz/23eF9vIp4Lr6/j+d6QAewFvg1nb2/vfh7Gc56dALwb+IEx5jJr7fPAD/D0iidZa3svXDPAdyRjzETva33fG8eXgX8aY8b54bkPAVvxfHf4nrd9ellru4CDQN84JUQpKZVg0tNbegWwD+gd7mKMMcAngP+01tZaa5vwfLDeAmCtrbHW/tNa2+q9727eTgx7/Mlae8Ba2wY8hie5Gogb+La1tsP7+E8C37DWlllrO4DvAO82nmGu64BV3sTvIuB/eTt5vNh7v68x/tlau9v7QfwOYI+19h/W2k7gZ8CJQdpwa5/H/wSIA1YC5wLjrLXftda6rLWHgd/3tJ/X69baf1lr3d5j7jVY+/fxLW+brcNzIuqvt7ITT2KZ770i35O0rwSSgHu8Mb4EPA2833uh4l3Af1trW6y1u4AH+uzzOuCotfZP1toua+02PAn9uwdpLxERGVg5nmRiMNcCxdbav3o/gx/Gcx6/3nu/G5hvjIm31lZ4e2LBcz7IByZYa9uttYMVTvofa22btfZNPBdZ+0tWOvEkatOttd3W2q3W2sZhxg19zsve8/+jeBJRvBc+C/Ccq/qTBjQNIbY/WWsPWWsbgOeAQ9baNd7vBH8HzhksbmPMJDzJ+1e9bboD+AOeZPdsBvqO9CHgWWvts97vB6uBLcA1I3muMWYynu8mPd8bXsFzYfp0Td52lBCnpFSCyV/xXG37KKcN3cXTq5aAZ35LvTGmHnjeux1jTIIx5nfGmBLjGX76CpDmTWZ69E3oWvEkQAM5aa1t73M7H3iiz2vvBbqBHGvtIaAZz4fwKjwnp3JvD19vUupjjMf6/D6h721v4tb3/v70fbybt6+U5gMTeuL3HsPX8fRU9vfapztr+3vVWWtb+twu8b726X6E58rni8YznOtr3u0TgGPeuPvuY6L3daJOi7Gkz+/5wIrTju+DeHpvRURkeCYCtT48bgKnfibjvT3Re154H55ewQpjzDPGmNnex3wFz7SUTcYzZeNjnJ0v5/G/Ai8Ajxhjyo0x/+sdiTOkuPvcPv3c+ADwAe/F2g8Dj3mT1f7U4elh9TW2yj6/t/Vzu+d4zxb3BKDn4vFAx9Sfgdo2H3jPaefXC/FcXB7JcyfQ//eG0yUD9YPELiFASakEDWttCZ6CR9cAj592dzWeD+R51to0779U6ynaAJ4hobOAFdbaFDw9lnCWOZiDhXPa7WN4hh6l9fkXZ98uXrAOT69cjHfbOjy9vum8PUzJlxj7vm4FniE+ngd5ToCTOLu+j4/AMxyq3Bv/kdPiT7bW9r3Sefox9zVY+wOke4dp9Zjsfe1TWGubrLVfstZOxXM1+ovGmMu8j51kTi0kMRlPj/lJoOu045/c5/djwLrTji/JWnvnWY5JREQG4P0svh7PkNfBlONJQPrq+fzGWvuCtfYKPMnIPjwjdbDWnrDWfsJaOwHPiKRfG2OmjyRu7wic/7HWzsUzVPg63q5Zcfp57qxx9/cca+0bgAvPRegPMPDQXYC38Ex78SW2oThb3OVAhjEmuZ/74Ozn+v4cA/562vk10Vp7zwifW0H/3xt6eUejTcfTKy4hTkmpBJvbgUtPu3LW0+v3e+Cnxphs6C0q1DMnMhlP0lTvnWvxbT/H9Vvg7p7J9saYccaYG/vcvw74LJ7eT4AiPHMiN1jPvJzhxPgMMM8Yc7P3g/k/GLznb2mfx38B6MAzX3cT0Gg8xZvijTGRxpj5xphzBz1yfGr/Hv9jjIkxxqzCc7L9++n7MsZcZzzFLQzQiKfHuRtPQYMW4CveebCFeL4QPeJtw8eB73h7nOdy6tyTp4GZxpgPe58bbYw5d4D5RiIiMgDv5+ccPPMSx+OZCjKYZ/F8Bn/AGBNljHkfMBd42niKDN3gTT468Iws6va+1nvM2wX/6vAkTN397H8o8V9ijFngHYXUiGfIbM8+K4GpvsQ9yMv8BU9NiK5Bhhw/S59pOoPENhQDxm2tPQa8Bvw/4ymstBDPd6ueWhGVQIHxvZLw3/AMC77K+90hzniKG+YN+syzPNfbEbGFt783XMipw6YBluOZmtNfD6qEGCWlElS8cym2DHD3V/EM+3zDO/x1DZ6eR/DMt4zH06P3Bp6hpf70c+ApPENOm7yvsaLP/evwJJ09SekGPMNdX+nzmCHFaK2tBt4D3APUADOAVweJ80k8w6Tq8Awrutl7ZbYbz4f9Yjy90dV45pikDrK/vs7W/uAZvlOH5yrtg8CnrLX7+tnPDO9zm/EUcfi1tbbIWusCbsAzl7Ya+DXwkT77+CyeIUEngD/jKf4AeHpfgSvxzHEt9z7mh4AW3RYR8c37jDHNeIZKPoXnvLPUWnvGiJfTWWtr8FyI/JL3eV8BrvOexyK828vxDAW+GPi096nnAhu9r/sU8Hlr7ZERHsd4PEX7GvFMtVnH24UTf46nHkSdMeYXg8R9Nn8F5nP2XlLwzJGcbYzpmcpytth85kPc78cz17UceAJPjYzV3vt6LhbXGGO2+fBax/AUQvw6nlFLx4D/woccwofnfgDPd6laPBfqT5+69UE8nQISBoxnmpqIjHXGmO/gKZ7wIQdeuxD4m7XWlyunIiIiY5bxLIdWBSyx1hYP8tg7gLnW2i+MRmyhwjsqax1wzmk1PiRERTkdgIiIiIjIGHInsHmwhBTAWnvfKMQTcqy1VYCm4IQRJaUiIiIiIj4wxhzFU6DwJmcjEQktGr4rIiIiIiIijlGhIxEREREREXGMklIRERERERFxTFDMKc3KyrIFBQV+2VdLSwuJiYmDPzCEhOMxQ3gedzgeM4TncYfjMYP/jnvr1q3V1tpxfggpbOncPLrURr5RO/lG7TQ4tZFv/NlOZzs3B0VSWlBQwJYtAy1NOTRFRUUUFhb6ZV9jRTgeM4TncYfjMUN4Hnc4HjP477iNMVpsfYR0bh5daiPfqJ18o3YanNrIN/5sp7OdmzV8V0RERERERByjpFREREREREQco6RUREREREREHKOkVERERERERByjpFREREREREQco6RUREREREREHKOkVERERERERByjpFREREREREQco6RUREREREREHKOkNMgcrGqmtKbV6TBEREREgs6h+m7qW11OhyEifqakNMh84dHtfP2JnU6HISIiIhJUXF1uvvdGO7f9ebPToYiIn0U5HYC8zVrLoaoWEmPbnQ5FREREJKhUNLQBsL203tlARMTv1FMaRE42ddDW2U11s4vaFg1NEREREelRVudJSiOMw4GIiN8pKQ0iR/vMJT1Q2eRgJCIiIiLB5Vit53tSdKS+voqEGv2vDiJHa1p6fy9WUioiIiLSq6en1KinVCTkKCkNIqU1rURGGJJiozhQ2ex0OCIiIiJBwVrLvhOeC/btnW6aO7ocjkhE/ElJaRA5WtPCpPR4ZuYkafiuiIiIiNe/dhxnzd5KYrzfXL/5xE6qGlUYUiRUKCkNIiU1rUzOTGRmTjLFVeopFREREQE4VNVChIGvLY8D4F87yrWEnkgIUVIaJKy1HK1poSAzgRk5ydS2uKhu7nA6LBERERHHNbV3khwXzdS0SF75r0u4al6OpjqJhBAlpUGirrWTpvYu8jMTmZmTBKgCr4iIiAhAY3sXKfFRAEzOTGB6dhLl9W10u63DkYmIPygpDRIl3sq7BZkJzMxJBqBYVwBFRGQIjDGzjDE7+vxrNMZ8wRiTYYxZbYwp9v5MdzpWkaFoau8kOTa69/ak9AS63JaKhjYHoxIRf1FSGiRKvGuU5mcmkJ0cS3JclHpKRURkSKy1+621i621i4GlQCvwBPA1YK21dgaw1ntbZMxobHu7pxRgUkYCAMdq305K3W7LN57YySU/LuK6X67nWG0rbvWkiowJPiWlxpijxpid3quuW7zbBrzqaoy5yxhz0Biz3xhzVaCCDyVHa1owBvLSEzDGeIodqadURESG7zLgkLW2BLgReMC7/QHgJqeCEhmORu+c0h6T0j1J6ft//wa1LS6OVLfwjX/t4sGNpUzJSqS0ppV3/vo15n77eZ7ccdypsEXER0PpKb3Ee/V1mfd2v1ddjTFzgVuAecDVwK+NMZF+jDkkldS0MiE1nrhoT1PNzEniQFUT1uoKn4iIDMstwMPe33OstRUA3p/ZjkUlMgxN7V2k9ElKJ6TFsTAvFYAl31vNJT8u4pHNpdx+4RT+eOsyPrFqKtXNHbR3uvn8Izu0rqlIkIsa/CEDuhEo9P7+AFAEfNW7/RFrbQdwxBhzEFgOvD6C1wp5R2tayM9M6L09IzuZh1uPcbK5g+zkOAcjExGRscYYEwPcANw1xOfdAdwBkJOTQ1FRkV/iaW5u9tu+QpXa6Oxqm9torDlBc0Jnbzt9cT78JSKKV493cf3UaFbkRjEuoYp166pIaOo+5fk/+fvLXJQX3c+eQ5P+nganNvLNaLWTr0mpBV40xljgd9ba+zjtqqsxpueq60TgjT7PLfNuk7MorWnlynnje2/3FDs6WNmspFRERIbqHcA2a22l93alMSbXe77OBar6e5L3/H4fwLJly2xhYaFfgikqKsJf+wpVaqOBud2W9heeZc70KSRFl5/SThddZHF1u3tHmvVY5bZ8741nee+yPLaU1LGnNZb/LjxvlCN3jv6eBqc28s1otZOvSekF1tpyb+K52hiz7yyPNf1sO2MMqq7Gvq2101LT4qK7voKiohoA6tvdADzz6nZcZWe/sjcWj9kfwvG4w/GYITyPOxyPGcL3uAPg/bw9dBfgKeBW4B7vzyedCErEV8/urGB8ahxLJqfT7OrCWkiJi4JTO0CJiDDERZw5SywywrD3u1cTExXB/z6/j/tfPUKrq4uEmCjW7KnkRGM7V87LGfaF/7V7K5mdm8LEtPhhPV9ETuVTUmqtLff+rDLGPIFnOO5AV13LgEl9np4HlPezT12N9dp1vAHWbuDS5QsonJ8LgLWWb29cjU0ZT2HhgrM+fywesz+E43GH4zFDeB53OB4zhO9x+5MxJgG4Avhkn833AI8ZY24HSoH3OBGbiC/cbsunH9wGwLULc7l4xjgAz5zSFt/3Ex/jSVYvmJ7F7145zEMbS5mZk8zH/7IFgPteOcwrX7lkyPGV1LRw+wOefWz71hVkJMYMeR8icqpBCx0ZYxKNMck9vwNXArt4+6ornHrV9SngFmNMrDFmCjAD2OTvwEPJUe8apfmZib3bPBV4kyjWsjAiIjIE1tpWa22mtbahz7Yaa+1l1toZ3p+1TsYocjYlta29v796sJqv/PMtAFLihzcn9NyCDBJjIvn+M3v5yP2er6QXzRxHRUPbsApKHql+OzN+s6x+WDGJyKl86SnNAZ4wxvQ8/iFr7fPGmM30c9XVWrvbGPMYsAfoAj5jre3uf9cCp65R2teMnGSeeasCay3e9hcREREJabvLPddTnv7chUzPTmLjkVoOnGji4pnj2Fh9thlk/YuPiWT1Fy+mtLaVzz28naWT01k4KZVXDpyko+vM+aiDKe2TNO+raOKSWSpmLTJSgyal1trDwKJ+ttfgWQOtv+fcDdw94ujCRElNC9nJsSTEnPp2zMxO4qG2Tk42dZCdomJHIiIiErq+/Pc3OV7XxuuHa4gwMCMnidioSC6eOY6LZ44b0b4npMUzIS2e1792KW4Lj245BkBjW+eQk9KSmlbioyNJT4hm34nGEcUlIh4jWRJG/ORoTesZvaTwdgXeA5XNSkpFREQkpP1ja1nv759YNZXYKP8vcx8V6Zm5lhLn+Qrc2N5FdsrAj69sbGdPRSMbD9eyeFIaW47W8scNR5iRnUR+ZgJbS+ro6nb37ldEhkdJaRAoqWlh1YwzrwDO6E1Km7hwRtZohyUiIiISECca2rnhVxv46+0rmDU++ZS5nb98/zlcv2hCQF8/Jc4zP7WpvXPAx3S7LZf/3zqaOrrOuC87JZZ3L53Ep/62lUc2H+NDK/MDFqtIONBlHYe1ubqpbOygoJ+e0qykGNIToimuUrEjERERCR2vHqymqqmDn64+AHh6LAG+ee2cgCekAMnentKm9jMTzh5N7Z00dXRxbkE6H/YmncmxUdz/0WV898b5XDUvh+zkWHYcqw94vCKhTkmpw3omy/etvNvDGMOM7GQOVDaPdlgiIiIiAZOW4Omp3HncU9SoprkDgKyk2FF5/eTentKBk9L6Vk8v6vuXT2b5lAwAYqMjuHR2DtPGJWGMITkuijaX6nmKjJSSUoe9vRzMmT2l4Jnkf6CyaVgly0VERESCUUeXG4Dj9Z5lWaqbXQBkJo3Omp9v95QOPHy3vs1zX2p8NAsmpgIwdVzSKY9JiImi1TVwYisivtGcUoeV9CSlGWf2lIKn2FFTexeVjR2MT1WxIxERERn7Orre7l38x9Yy/usfnrVIMxNHq6d08OG7Dd6kNC0hmvzMBO65eQGXzj51+Zf4mEha1FMqMmLqKXXY0ZpW0hOiSU3of0HoGTmeK3IHKjWvVEREREJDR6e79/eehBQgK3l0ekoTY6KIMIP0lLZ6em9T42MwxnDL8slnrIaQEBOp4bsifqCk1GElNS39zift0bMsTHGV5pWKiIhIaOgZvnt6oceMhNFJSiMiDEmxUb0FlvrT2Gf47kASNXxXxC80fNdhJTWtLMtPH/D+rKRYMhJjKFZPqYiIiISI9k5P7+LDd6wkOS6ae18+yAu7Tozqep/JcdE0DtBT2ubqZktJHXD2pDRePaUifqGeUgd1dHVTXt/G5LP0lALMyE7S8F0REREJGT09peOSYkmKjeKrV8/mpS8XjmoM2SmxlNe39Xvf//x7N0/uKAcgJmrgr8sJMZG0diopFRkpJaUOKqtrw23PHLpyupk5yRRXNqsCr4iIiISEjq5uoiLMqPaMnm5mtuf7VX/2nfCtMyA+JpLWDiWlIiOlpNRBvZV3B+kpnZmTRFNHFyca20cjLBEREZGA6uh0E3uWHsjRMCMniZoWF9XeNVL7GqzDoEdCdBSubjdd3e7BHywiA1JS6qCj1a3A4B98M7zFjg4McDVPREREZCxp7+omNjrS0Rhmje/5fnVmr2izt/fz9CVgTpcY6zkGDeEVGRklpQ4qrW0lKTaKjMSzV5rrrcCreaUiIiISAoKhp3RObgoRBtburaKz201nn97OxrZOVk7N4P6PnnvWfcTHeJJSFTsSGRlV33XQ0ZoW8jMTMMac9XEZiTFkJcWo2JGIiIiEhI4u55PSrKRYbl6Sx1/fKOHfb5ZT1dTBxLR41nzxYurbXEzNShp0HwnepLRVSanIiKin1EElNa0UDDKftMeM7GQN3xUREZGQ0NHVTZzDw3cBPrhiMq4uN1VNnnmlx+vbOFzdTH1rJ2kJAy8F0yM+2tO/09KhtUpFRkJJqUO6ut0cq20l38eJ9DNykjhYpQq8IiIiMvYFQ08pwIKJqb2///R9iwBPzY+Gts6zrk/ao2dOaZvmlIqMiPOfBmGqvL6dLrf1vac0J5nmji7KG1SBV0RERMa29s5uYqOc7ymNiowg3ttje/mcHAD2n2iko8tNqg89pRq+K+IfmlPqkJJaz3Iwk33sKZ2Z7ZnXcKCyiYlp8QGLS0RERCTQOrrcJMUGx9fQNV+6mNpmF8lx0YxLjmVHWQMAafFnL0QJkBDjOYaj1S2cPy2TaAfXXRUZy/Q/xyFHa3qWg/Gtp1QVeEVERCRUBEP13R4T0+JZkOcZxjslM5FNR2oAfBq+Oz07idnjk/n2U7u57U+bAxqnSCgLjk+DMFRS3UJcdATZybE+PT49MYaspFgVOxIREZExYf+JJn65tphdxxvOuK8jCNYp7c9Hzs9nwcRUZuYkMX9iyqCPj46M4PcfWQbA5qO1qv0hMkzBMW4iDB2taSU/I5GIiLMvB9PXzJwkiquUlIqIiEhw23+iiet/tQFXl5v1B6t57JPnAbC3opGn3yrn0MkWFk1KczbIfly3cALXLZwwpOdMykjgW9fN5XtP76GutXPQ9edF5ExKSh1SWtvi89DdHjNzkvn7lmNYawdd21RkNPzkxf28eqiGf955vtOhiIhIEHmrrB5Xl5ur5uXwwu5KfvTCPt44XMvWkrrex0RHhM6AvQJvjZCjNS1KSkWGIXQ+DcYQt9tSUuP7cjA9ZuQk0eLq5nh9W4AiExmaF/dUsrWkjoPqwRcJGsaYNGPMP4wx+4wxe40x5xljMowxq40xxd6f6U7HKaGtpxrth1bmA3Dvy4eobu7gPy6dzpevnAnAoZOhc+7I93Y0lNS0OByJyNikpNQBlU3tdHS5ez/AfPV2saPQ+RCXsaupvZP93sJbL+454XA0ItLHz4HnrbWzgUXAXuBrwFpr7Qxgrfe2SMD0JKXL8jN46BMr+NvtKyj6ciFfvHIW7102CQittT0nZcQTYeC7/97Dxx/YQrdbc0tFhkJJqQOOVg+t8m6PmdmepPSAKvBKENhxrB5rITYqgtV7Kp0OR0QAY0wKcBHwRwBrrctaWw/cCDzgfdgDwE1OxCfho9XVhTEQFx3B+dOyuHBGVu/Uo+yUOH78nkX86gNLHI7Sf2KjIvmfG+axeFIaa/ZW8vqhGqdDEhlTlJQ6oGdox1CH76YmRJOdrAq8Ehy2ldRjDNx6fgHbS+upamx3OiQRganASeBPxpjtxpg/GGMSgRxrbQWA92e2k0FK6Gvp6CYxJmrAGhjvXprHlKyhXZwPdh8+r4DffGgpybFR/HbdIRpaO50OSWTMUKEjB5TUthIdaZiQFj/k587MSaa4Sj2l4rxtpXXMyE7iXUvyuO+Vw6zZW8UHVkx2OiyRcBcFLAE+Z63daIz5OUMYqmuMuQO4AyAnJ4eioiK/BNXc3Oy3fYWqUGujgyUdRNHt92MaC+103ZQIHtlXzap7VnPDtGhWl3RSOCmaaWkRxEcZJiUHvk9oLLST09RGvhmtdlJS6oCSmhYmpScQOYTlYHpMz07i0c3HcLvtkJaTEfEnt9uyvbSOaxbkMjMnickZCby454SSUhHnlQFl1tqN3tv/wJOUVhpjcq21FcaYXKCqvydba+8D7gNYtmyZLSws9EtQRUVF+GtfoSrU2uifFdtJb2/w+zGNhXYqBD54vIGvP7GTv+31rNH66H5X7/1/vX05q2aMC2gMY6GdnKY28s1otZOSUgccrR565d0eM3OSaev0VOCdlDG8fYiM1OHqZhrbu1iSn44xhivn5vCX10to7ugiKVYfKyJOsdaeMMYcM8bMstbuBy4D9nj/3Qrc4/35pINhShho7egiISbS6TAcM39iKk98+gL+ubWMrOQYqho7aHV1892n9/DyvpMBT0pFxhp9exxl1lpKalpYPiVjWM+fmZMEeIodKSkVp2wrqQdgyWTPqhJXzM3hDxuOsG7/Sa5dmOtgZCICfA540BgTAxwGbsNTQ+IxY8ztQCnwHgfjkzDQ4uoiMSa8v2ZGRhjee+6kU7Y9u7OCN8vqnQlIJIip0NEoq2520eLq7l1keahm5PRU4A2dYkc7yxq46qevUKlCOWPGttI6UuOjmeotUrE0P52MxBhWa2kYEcdZa3dYa5dZaxdaa2+y1tZZa2ustZdZa2d4f9Y6HaeEtjZXNwmx4dtTOpCFeWnsLm+gq9vtdCgiQUVJ6Sgrre2pvDu8inOp8dHkpMRSHELLwvxszQH2VzZRtL/fKU4ShLaV1nHO5LTeec1RkRFcOjubl/ZV0akTrYhI2GtxdYd9T2l/5k1Iob3TTUltq9OhiAQVJaWjrGeN0uHOKQXPvNIDIVKBd/+JJtbu8ySjGw/rwv1Y0NjeSXFVc+/Q3R5Xzs2hsb2LTUf0PoqIhLtwn1M6kLx0z8oLx+vaHI5EJLgoKR1lJTUtRBjISx9+UjojO5mDVc243daPkTnjd68cIj46kvOnZbJRycyYsKO0Hms5IyldNWMccdERvLhbQ3hFRMJdi6ubRBW+O0Oetx7I8XolpSJ9KSkdZUdrWpmYHk9M1PCbfmZOEu2dbsrG+FW24/VtPLWjnFuWT+KqeeM5Xt/GMQ1nCXpbS+owBhZNSj1le3xMJKtmjGP1nkqsHfsXTEREZPhaXV3Eq6f0DDnJsURGGPWUipxGSekoK6lpIT9jePNJe7xd7GhsD+H9w/rDAHx81VRWTPVUI1ZvqX8crGrie0/v4eqfvcKhk/4tirWttI5ZOckkx0Wfcd8Vc3Mob2hnd3mjX19TRETGDleXm85uS6KS0jNERUYwPiWOsjpdhBfpS0npKCupHf4apT1m9CwLM4bnlda1uHhk0zFuWDyBiWnxzMxOJi0hmo2Ha5wObcxqc3Xzz61lvPs3r3H5T17hL68fZX9lE09uP+6313C7LTuO1XPOaUN3e1w2O5sIg4bwioiEsfo2FwAJKnTUr4np8Rq+K3Ian5NSY0ykMWa7MeZp7+0MY8xqY0yx92d6n8feZYw5aIzZb4y5KhCBj0X1rS7qWzspGGbl3R4pcdHkpsZRPIaXhfnL6yW0dXbzqYunARARYVhekKGe0mHYW9HIt5/cxfIfrOFLf3+TmhYXX79mNq/fdRnL8tNZs9d/VY0Pnmymqb2LJZPT+r0/MymWZfkZvLin0m+vKSIiY4e1lqL9JwFYVtD/BcxwN29CCltL6li7V+dKkR5DuYT1eWAvkOK9/TVgrbX2HmPM17y3v2qMmQvcAswDJgBrjDEzrbXdfox7TCqpGXnl3R4zcpLH7PDdVlcXf37tCJfNzmamdygywIqpmby4p5KKhjZyU+MdjDD4tXR08fRb5Ty06RhvHqsnJiqCd8wfz/uXT2bFlAyM8SzVctmcHO55bh/l9W1MSBt5m24rqQNgSf7AXzSunJfD95/Zy7HaViZljPxvXURExoZ9Jxp5929ep7mji8kZCSyYmDr4k8LQl6+cxdaSOj7z0Dae+uyFp3wXEglXPvWUGmPygGuBP/TZfCPwgPf3B4Cb+mx/xFrbYa09AhwElvsl2jHuaI1njdKCrJH1lALMzE7iYFUz3WOwAu9jm49R19rJnYXTTtm+Yop3XqmWhhnQ0YZu7np8J8vvXsNX/7mT1o4u/vu6uWy86zJ+fss5rJya2ZuQAlw+Jxugd9mdkdpWWkdaQjRTz/I3fMXcHAD1loqIhJnNR+to7ugC4FMXTzvlfCRvS4yN4o+3nkukMfym6JDT4YgEBV+H7/4M+Arg7rMtx1pbAeD9me3dPhE41udxZd5tYa/U21M62Q+9RzNykujoco+5arWd3W5+v/4Iy/LTWVaQccp9c3JTSI6LYuMRzSvtz0MbS/nO6+08sb2MdyzI5Z93nseL/3kRH7twCumJMf0+Z9q4JPIzE/w2RGhbaT1LJqef9YtGfmYis3KSNa9URCTMVHjnSf74PYv4wIrJDkcT3MYlx/LupXn8+81y2lxhP5hQZPDhu8aY64Aqa+1WY0yhD/vs79vqGd15xpg7gDsAcnJyKCoq8mHXg2tubvbbvvztjd0dpMca3nh1/Yj31VTv+QB7fO3rzExsD9pjPt1r5V0cr+/g3VPd/cY8Ndny8q4yijIG7y0N5vc6EB7c3EZOvOW/z48jMbqOpiN1rDsy+PNmJbl4qbiVF9a8TGzU8K9at3RaDla1sjC1Y9B2n5nk4ulDTTz94sskxYz8Snm4vdcQnscM4XvcIqHgWF0b+ZkJvHtpntOhjAkzxyfT5bY0tndq+RwJe77MKb0AuMEYcw0QB6QYY/4GVBpjcq21FcaYXKBnfGAZMKnP8/OA8tN3aq29D7gPYNmyZbawsHD4R9FHUVER/tqXv/1q72vMnGAoLDxvxPta2t7J9994kdjsApJMWdAec1/WWu75+XpmZEfz+XdfRETEmclKccRh7n52L3OXrCQ7Je6s+wvm99rfOrvd3Ln2RS7IjebaKy4Z0nOj86p58Q8bMblzKJw3ftgxFO2vAjbz7ouXcP70rLM+NmN6Pf/+1au0Z87gOj98OQmn97pHOB4zhO9xi4SCY7WtTEpXLQFfJXgT0Vb1lIoMPnzXWnuXtTbPWluAp4DRS9baDwFPAbd6H3Yr8KT396eAW4wxscaYKcAMYJPfIx+Djta0jrjybo/kuGgmpMaNqWJHRftPsu9EE5+6eFq/CSmg9UoHsOt4A22d3czKGPqV1HMLMkiOjWLtCKvwbiutJ8LAoklpgz52wcRUxqfEsXqPhvCKiIQLFbgbmp4lc1q883BFwtlI1im9B7jCGFMMXOG9jbV2N/AYsAd4HviMKu9Cc0cX1c0d5Gf578PaU4F37CwL85t1h5iQGscNiycM+Ji5uSkkxWpe6ek2eZP0melDT0pjoiK4aNY41u6rwj2CwljbSuqYNT6FxNjBB1gYY7hibg6vHKimvTPs//uLiIS8lo4ualpcTMpQ9Xxf9fSUtuk8KTK0pNRaW2Stvc77e4219jJr7Qzvz9o+j7vbWjvNWjvLWvucv4Mei3qKHOVn+KenFGBmThKHTjbjtsFfgXdrSR2bjtRy+6qpREcO/GcXFRnBsoJ0VeA9zaYjtUwdl0hq7PDmZ14+J5vq5g7eOt4wrOd3uy07jtUPuD5pf66cl0NbZzcbiquH9ZoiIjJ2lNV5ihxp+K7vNHxX5G0j6SmVISjxLgfjjzVKe8zIScbV5aaqNfiT0t+uO0RaQjS3nDtp0MeumJJJcVUz1c0doxBZ8Ot2WzYdre1dMmc4CmdmE2HgpWFW4S2uaqK5o4slk31fCH3FlEySY6N4UUN4RURCXmmt/1YYCBfx0Z6RR60aviuipHS0HO3pKfVjUtqz2PLxZvcgj3zbyaYONhRXj+r6pgermli9p5KPnFfg09DPnnmlmzSvFID9J5poau9i+QiS0vTEGJbmp7NmmPNKt5XUA7Ak3/ekNCYqgktmZ7N2b9WYXE9XREQGZ72jtXqWqNOcUt8lxqqnVKSHktJRUlLTQlZSDMlx0X7b54zsJODsSWmbq5ui/VXc/cwerv7ZK5x79xo+9MeNPL9r9HqvfrvuMHHREXz0/AKfHr9gYioJMZFsPKx5pUDv/NrlUzJHtJ/L5uSwp6KRcu86ckOxrbSOjMQYCoZ4UeWKuTnUtLjYVlo35NcUEZHgVt3cweU/WceXHnuTl/ZVkRgTSXqC/77nhLqeZWBaNadUxKclYcQPjta0+H1IS2JsFBPT4ilvdvVu63Zbdh1vYMPBajYUV7O1pA5Xt5uYyAiW5qfzX1fN4lcvHWTz0VquXZjr13j6U9HQxpM7jvPBFflkJMb49Jxob6yqwOux6UgtE9PimZgWT/EI9nP5nGzueW4fa/dV8eGV+UN67rbSOs6ZlIYxQ5vTWjhrHNGRhtV7Kjm3YPg9vSIi4jxrLWV1bWQmxfDPbcf5w/rDlNS0cuikZ4pSfHTkkM8T4ayn+q6G74ooKR01pTWtrJw6sp6u/szMSWLvsWoe3FjCqwerefVgDQ1tnQDMyU3h1vPzuXDGOJYXZPReketJVkfDH9cfwW3h9gunDOl5K6Zk8OMXD1DX4iLdx2Q2FFlr2XSklotnjhvxvqaNSyI/M4G1eyuHlJTWt7o4fLKFdy0Z+nqjyXHRnDctixd2n+Cud8zWlxURkTHs4U3H+PoTO3tvL8xL5f6PLqO+tZPNR+uYNyHFwejGnvhoDd8V6aGkdBS0d3ZT3tBOvp/WKO1r1vgUXt5/km88sYvc1DiunJvDhTOyOH9aFuOSY/t9ztL8dH6z7hAtHV0+zfEcrvpWFw9vKuX6hblDnmOywpvAbzpay1XzxgcivDHh0MkWalpcI5pP2sMYw6Wzs3lwYymtrq7eK7SD2V5aDzCkIkd9XTk3h2/+axfFVc2986BFRGTseXLHccBzQfx7N85n+ZSM3ouNNw/jwmW4i4wwxEVHaEkYEZSUjoqeyf8FflyjtMfHLizAVXOMD1x5HtPGJfrUE7U0P51ut+XNsnrOn5bl95h6/PX1Elpc3Xzy4mlDfu7CvFRioyLYeDi8k9KeYk/+SEoBLp+Tw59ePcqG4mqu9LFdt5XWERlhWDQpdViveYU3KV29p1JJqYjIGLS3opG/vF7CxiO1fOriaXz16lka+eInCTFRtGj4rogKHY2Gtyvv+r+nNDs5jovyopmeneTzCeIc71qT2wI4hLe9s5s/v3aUS2aNY07u0IfzxEZFsmRyem+Rn3C16UgNWUmxTMnyz9/OuQUZJMdGsXYIVXi3ldYxe3yyzz2rp8tJiWPRpDRe3K2lYURExqL7XjnMw5tKuWjmOG67oEAJqR/FR0fSpuG7IkpKR0PvGqVBUiY9LSGG6dlJAZ1X+vctx6hpcfGpYfSS9lgxNYM9FY29c2TDjbWWjUc865P66wtATFQEF80ax9p9Vbh9WKal223ZUVo/7KG7Pa6cm8ObZQ2caGgf0X5ERGT0lda2snJqBn/52HJyUuKcDiekJMREak6pCEpKR0VJTSspcVGkBVGZ9KWT09lWWu9TYjJUXd1ufvfKYc6ZnDaiYacrpmRiLWw5Gp5VeMvq2qhoaO9dt9VfLp+TTXVzB28dbxj0sftPNNHi6mZJftqIXvPKuTkArN5bOaL9iIjI6Cura2VSenBcWA81CbFRtLg0fFdESekoOFrTQkGWb/M9R8vS/HQa2jo5XN3s930/s7OCsro27rx42oiO+ZzJacRERoTt0jD+nk/ao3BmNhEGXvIhQexZX3SkPaXTs5MoyExg9R4lpSKBZow5aozZaYzZYYzZ4t2WYYxZbYwp9v4c2X9qCRvtnd1UNnaQp6Q0IBKi1VMqAkpKR0VJTWtA5pOOxJJ8z/eRQAzh/df240zKiOfyOTkj2k9cdCSLJ6Wx8XDwzys92dTB9b/c4NdYNx2pJTU+mpnZ/i0OlJ4Yw9L8dNb4MK90W2kdmYkxI15j1xjDlfPG8/qhahrbw3M4tsgou8Rau9hau8x7+2vAWmvtDGCt97bIoA5UNgEwKSPe4UhC07TsRLaV1vH5R7bzzX/t5MIfvsSdf9tKTXOH06GJjColpQHW2e3meH1b0Mwn7TFtXCJpCdF+T0o7urp543Atl8zKJiJi5D3DK6ZmsKu8keYgr0z3i7XF7DzewP2vHvHbPjcdreXcggy/tOPpLpuTw56KRsrr2876uO2l9ZwzOd0vvfxXzM2hs9uybv/JEe9LRIbsRuAB7+8PADc5F4qMFcdqW7nhV68CqKc0QL5xzVzev3wyT+4o529vlJKVFMtL+6q45hfrue+VQ3QHYJqVSDBSUhpgx+va6HZb8jOD68PcGMPSyel+T0q3ldTT1tnNhdP9s9TMiimZdLttUM8rPXSymYc2lZIUG8VL+6qob3WNeJ9Vje0cqW5hhZ+H7va4fE42AGv3DdxbWtvi4kh1y4jnk/ZYMjmdzMQYXtQQXpFAs8CLxpitxpg7vNtyrLUVAN6f2Y5FJ2PGr146CMCls7NZmDe8ZcHk7OJjIvnBOxfw/BdW8b0b5/H4nefz59uWMzkjgR88u4+P3L+Rrm6302GKBJzWKQ2wo97KuwV+WtLDn5bkp7N2XxV1LS7SE2P8ss8NB08SGWFYOS3TL/tbkp9GVIRh45FaCmcF53eoHz2/n7ioCO794BJuvX8T/36rgg+vzB/RPjcdDcx80h7TxiUxOSOBtXsrB4x1u3c+6dIRziftERlhuGxONs/tPIGry01MlK6JiQTIBdbacmNMNrDaGLPP1yd6k9g7AHJycigqKvJLQM3NzX7bV6gKxjbasK+Vc7Ij+UhBC2+8ut7pcIDgbCd/mQS88spRAD49y/KvyGiePFjDI88WkZc8tHNmKLeTv6iNfDNa7aSkNMBKetcoDa6eUvAUOwLYfqyOS2ePbP5nj/XF1ZwzKY2UOP9UGk6IiWJhXmrQzivdWlLL87tP8MUrZnLRjCxmj0/m8W1lI09Kj9SSEBPJvAlDX+PVF8Z4EsQHN5bS6urqdw3SbaV1REUYFual+e11r1mQy2NbyvjH1jI+sGKy3/YrIm+z1pZ7f1YZY54AlgOVxphca22FMSYX6HeYhLX2PuA+gGXLltnCwkK/xFRUVIS/9hWqgrGNzMaXmDIxg8LCxU6H0isY2ylQunIqefLQFhads5QFQ+ypDqd2Gi61kW9Gq53UVRFgR2taSIiJZFxSrNOhnGFRXhqREcZvQ3jrWlzsPN7AhTP8M3S3x4qpmbxV1kBrkJVMt9byg2f3MS45lo+vmoIxhneeM5HtpfUcqW4Z0b43HallaX46UZGB+y96+ZwcXF1uNhRX93v/tpJ65uSmEB8T6bfXvHjmOM4tSOcnq/cH/TxhkbHIGJNojEnu+R24EtgFPAXc6n3YrcCTzkQoY0mbq9uv5wAZmljviCJXt6rzSuhTUhpgJTWtTM5ICKrlYHrEe3vithz1T1L62qEarIVV/k5Kp2TQ5bZsK6n3635H6oXdJ9haUscXr5jZ29N40zkTiTDwxLayYe+3vtXFvhNNAZtP2uPcggySY6NY208V3q5uN2+W1bNkcppfX9MYwzeunUt1s4vfFh3y675FBIAcYIMx5k1gE/CMtfZ54B7gCmNMMXCF97bIWbW6uklQUuqYnmkuHZ2aUyqhT0lpgJXUtFAQZMvB9LVkcjpvltXT6YdJ9OuLT5IcG8UiPw73BFhWkEFkhGHjkeAZwtvZ7eaHz+9nenYS71ma17s9JyWOC6Zn8fj247iHWTFvs/ciwfIp/pmXO5CYqAgumjWOtfuqzoh1f2UTra7u3qWD/GnxpDRuXDyB368/PGj1XxEZGmvtYWvtIu+/edbau73ba6y1l1lrZ3h/Bm/1OAkKbrelrbOb+GglpU7pTUpV6EjCgJLSAOp2W47VtpGfFXzzSXsszU+nvdPN3orGEe3HWsv64mrOm5bp9yGnSbFRzJ+QwsbDwfMd6pFNpRypbuGud8w+43hvXjKRsro2tgxzWPSmIzXEREWMSqXDy2ZnU93cwc7jDads3+aNfYmfihyd7r+umoUFfvTC/oDsX0RERqa9yzNkNL6fmgMyOmK83y9cXUpKJfQpKQ2gk00duLrdTAritb16ih2NdF7pkeoWjte3+X3obo8VUzPZcaye9k7n51U0d3TxszXFrJiSwaWzz6wIfNW88STERPL4MIfwbjpSy+JJacSNwtXpS2ZlE2Fg7d5Tl2nZVlpPVlIseemBWSw9Lz2B2y+cwhPbj/NWWX1AXiOQtpfWqUS/iIS0VpfnfKvhu87pnVOqpFTCgJLSAKpu7gAgKwiLHPWYkBZPbmrciJPSDQc9xXJWzRjnj7DOsGJKBq5uN9tL6wOy/6G4b90halpc3HXNnH7nCifERHH1/PE8s7NiyEl0c0cXu8obAz6ftEd6YgxL89NZc9q80m2ldSyZnBbQudCfLpxGZmIM339mL9aOncXBNx2p5Z2/fo2/vVHidCgiIgHT5urpKVVS6pQYJaUSRpSUBlBNiwuArCT/rAEaKEvy03uHaw7X+uJq8tLjA7b0zbKCDIzB8XmllY3t/H79Ea5bmMviSWkDPu5dS/Joau9izWk9kIPZVlJHt9sGbH3S/lw2J4c9FY298zurmzsoqWkNyHzSvpLjovnCFTPZdKSWF/cMrZ2c9McNhwH491sVDkciIhI4bZ3qKXVabJSn7TuUlEoYUFIaQLUtnp7SzCDuKQVYlp9OeUP7sIvOdHa7ef1QDatmjAtYz1pqfDRzc52fV/qzNQfocrv5ylWzz/q4lVMzyU2N4/Ftx4e0/41HaoiMMAGby9mfy7xDkNfu8/SW9vRGj0YM7z93EtOzk7jnuX1j4kpwaU0rL+6pZFxyLFtL6lSoSURClobvOu/tnlLnpy6JBJqS0gCqafb0lGYkBndPac+80m2lw+stffNYPc0dXQGbT9pjxZRMtpXW0eHQh/OByiYe3XyMD68sYPIgPcKREYYbF09k3YGTvcO4fbHpSC3zJ6aSGDt6hSWmZycxOSOBl7y9uttK64iKMKNSaCkqMoKvXzObI9UtPLgx+IfDPvD6USKN4ZfvPweAZ9RbKiIhqmdt8PhoFTpySm9SqhoGEgaUlAZQdbOL6EhDSlxwf6DPyU0hLjpi2PNK1xdXYwycPy2wS5ismJpBR5ebt8oaBn9wAPzwuX0kxkbxuUun+/T4m5dMpNtteWpHuU+Pb+/s5s1jDaM2n7SHMYbL5mTz6qEaWl1dbCupY96ElFEptASeYksXTs/i52uLaWjtHPH+qhrbqWxs90Nkp2pq7+TRzce4ZkEuK6dmsmBiKk+/5dt7KyIy1rSpp9Rxqr4r4URJaQDVNHeQmRgb0GIx/hAdGcGivLRhzytdX3yShXlppCUEtkd4eYEnWdt4ePTnlb5+qIa1+6r4dOF00n3s+Z6Zk8yCiak8vt23Krw7jtXj6nb3HudounxODq4uN+v2n+StsgbOGcXhw8YYvn7NHBraOvnlS8Uj2tfrh2q47CfreNdvXvN7j/o/tpbR3NHFxy6cAsB1C3N5s6yBY7Wtfn0dEZFgoOG7zouO9Hx/VFIq4UBJaQDVtriCfuhuj6X56ewub+y9MuqrxvZO3ixrYNX0wA7dBU+l2Nnjk9l4ZHTnlbrdlv/33F5yU+O47YKCIT33nedMZNfxRg5UNg362E1HajEGznUgKT23IIPk2CjuLTpIW2d3wIscnW7uhBTeszSPB14/SklNy7D28dSb5dx6/yYSY6Ioq2vj4Y2lfouv223582tHWTI5rbfA1bULcwF4WkN4RSQEqfqu84wxxEZFqNCRhAUlpQFU3eIiM8gr7/ZYmp9Ol9sOec3I1w/V0O22XBjg+aQ9VkzJYGtJHZ2jOL/imZ0VvFXWwJeunDXkIa03LJ5AZITxqeDRpiO1zB6fQmpC9HBDHbaYqAgumjmOXccbAVgyOW3UY/jSlbOIiojgh8/vG9LzrLX8bt0h/uPh7SyelMbzX1jF+dMy+eVLB2nu6PJLbC/tq6KkprW3lxQ8a60unpSmIbwiEpLenlOqpNRJMUpKJUwoKQ2gmuaOoF6jtK+e4ZpbhjiEd33xSRJiIketWuyKqZm0urrZeXx05pV2dHXzvy/sY/b4ZN55zsQhPz8rKZaLZ47jX9uP0+0eeC3Ozm43W0vqRn0+aV+XzfFU4c1OjmViWvyov35OShyfvHgqz+48wZajvvWGd7st33lqN//vuX1cuzCXv9y+nLSEGL5y9WxqWlz8cf0Rv8R2/4YjTEiN4+p540/Zft3CXHaXN3Kkeni9u2OFtZaHNpbykxf3j+oFIRFxTmvvkjDBXRcj1MVGRajQkYQFJaUBVNviInOMDN/NSIxh6rjEIc8r3VBczcqpmb0V4gKtZ/3O0Voa5sE3SjlW28bXr5lDZMTw5gbfvGQiJxrbeeMsc2F3HW+grbN7VNcnPd0ls7KJMJ6lYJyaB33HRVPJSYnl+8/sxdqBk3jwDC27829beeD1Ej6xagq/vOWc3p7sxZPSeMf88fx+/WFqhlD9uD97yht5/XANHzm/gKjIU//Oe4bwPhPCvaXNHV189uHtfP2JnfzipYN8+I8bqfWuwSwioavN1Y0xEBetr4pOiomM0JxSCQv6pAmQVlcXra5uMsbI8F2ApZPT2VpaN2gy0ONYbStHa1q5cBTmk/bISoplenYSG48EvthRT+GdVTOyuGjmuGHv5/I5OSTHRfHPbQMXPNrknSfrxHzSHumJMfzwXQv5rI/VhQMhISaKL105ix3H6vn3WeZq1ra4+MAf3mD13kq+ff1cvnHtXCJOu2jwpStn0erq4t6XD40opj+9eoT46EhuOXfSGfflpsZzbkF6yM4r3X+iiRt+tYHndlbwtXfM5qfvW8S20npu+NUG9lY0Oh2eiASItZaNh2vJTYkL+mKNoS4mSkmphAclpQHSs0ZpVuLYGL4Lnnml9a2dHPZxKOL64moALpo5ekkpeOaVbjlad9bhsP7wm6JD1Ld18tWrZ49oP3HRkVy7IJfnd53onaNzuk1Hapk6LpFxyc7+vbxn2STmTwz8+qRn864leczNTeGHz+2jvfPMwlslNS286zevsae8kd98cAm3XTCln7141l99z9JJ/O2NEsrqhlcht7q5gyd3lPOupRMHrC597YJc9p1o4mDV4MWsxpLHt5Vx470baGzr4qFPrORTF0/jnefk8fdPnkdnt5t3/eY1nt8Vmsm4SLjbUlLHpqO13Fk4zelQwl5sVKRj67OLjCYlpQFS4x3eNlYKHYEnKQV8Xq90w8GTjE+JY9q4pECGdYYVUzNp7uiitClwVw7L69u4/9UjvHPxRL8kaTcvyaPV1c0Lu0+ccV+327LpaK2j80mDSWSE4ZvXzuF4fRt/fu3oKfftOFbPzb9+jbpWFw99YgVXz889676+cMUMMPDT1cNbaubBN0pxdbv56Pn9J74A1yzIxRj495uhkaC1d3bz9Sd28sXH3mRRXhrP/seFrJz69hrEiyal8e/PXsjMnGQ+9bdt/HT1AdwBvkAkIqPrtYM1GAM3LB56LQXxL/WUSrhQUhogtS2eeWxjZUkYgGnjkkiNj/ZpXmm32/LqwRounJE16kN7VnqTt721gbty+MjmY3R1u/nilTP9sr9l+elMyojvtwrv/hNNNLV3OTqfNNicPz2Ly2Znc+9LB3vnhK7ZU8kt971OQmwk/7zzfJbmD95euanxfPT8Ah7fXsb+E0Pryezo6uavb5RQOGsc07MHvvCSnRLHiikZPLOzwueh74Opa3Hx1skuTjaNbD7sUB2rbeXdv32NhzaWcmfhNB78+AqyU+LOeFx2ShyP3LGSdy3J4+dri/n0g9to8VOlYxFxhrWWF3ef4NpfrOenaw4wMzuZ1PjRrwYvp4pRoSMJE0pKA6S6Z/juGKm+CxARYVgyOc2nntKdxxtoaOtk1SgtBdNXdkocs8cn82ZV4JLStXsrWZqfTl56gl/2FxFheOfiibx6sJoTDe2n3LfJOz92+ZTM/p4atu66Zg6tnd38bE0xL5V2csdftzAzJ5nH77xgSL3zd148jaSYKH784v4hvf4zb1VQ3dzBxwYYHtzXdQsncLCqmf0+rEfriy8+toOfbO3g3LvXcOEPX+JzD2/njxuOsK20LmDDuNbsqeTaX6yntKaVP3xkGV+9evYZhZ36iouO5MfvWcg3r53Di3tO8K7fvMax2uENkxYR5+w70ciu4w3cdO+r3PHXrVQ2es5RkzL8c/6TkVGhIwkXSkoDpGdO6VgavgueIbzFVc00tHae9XEbik8CcMEoFjnq64q5ORTXu6kLQBXQioY2dpc3ctmcHL/u951L8nBbeHLHqb2lm47WMjEt3pFlWILZ9OwkPrB8Mn/bWMJf9rgonJXNI3esHPK82/TEGD5VOI3VeyrZWuJb1WZrLX/ccITp2Uk+XXi5ev54Igw87YchvK8drObl/Se5Ij+Kb1wzh4V5qWw9Wsv3nt7Dzb9+jQXffpEb732V7zy1myd3HKe0pnVEPbRd3W7ueW4fH//LFiZnJvD051Zx+Vzf/vaNMXx81VT+dNtyyuvbuPHeV89aZVpEgsP20jre+etXeW5nBVf/bD3X/XIDZXVt/O+7FvL6XZfxzWvncNc1I6unIP4RExXBlpI6XjtUrakSEtK0+FSA1LZ0EB8dOebW91rinVe67Vgdl8zKHvBx64urmZub4lhP8OVzcvjlSwcpOlDFO8/J8+u+1+6t8r7GwMc/HFOyElkyOY3Htx3njoumYozBWsumI7VcNGP41X1D2Rcun8FL+6qYldzJfR9eetaeu7O57YIC/vTqUX743H4e/eTKQYecbz5ax+7yRn7wzgU+DU/PSorl/GlZPP1WOV+6cuawh7S73Za7n93LxLR43jPTcOVFU3vvq2xsZ3tpPduP1bGjtJ5HNx/rnXObmRjDwrxU8jMTyU2NY0JaPBPSPD+zk+MGXM6oqqmdzz20nY1Hann/8sl8+/q5vcvqDMXFM8fxr89cwCf+soUP/WEj375hHh9emT+sNhCRwPvjhiNsL63nzge39W67YfEE3uutMv7xVVMHeqqMsujICKyFD/x+IwvzUnngtuWkj6GpYSK+GlsZ0xhS0+waU/NJeyzKSyMywrCtZOCktKWji22ldXzswsGHNQbKgomppMUa1uwJRFJaSX5mQkAKOL1zSR7f+tcu9lQ0Mm9CKoerW6hudmk+6QAyk2LZ8NVLWLdu3bATUvAsNfP5y6bzrSd3U3Tg5FkvuADcv+EIaQnRvPMc34t8XLcwl689vpPd5Y3DLo715JvH2V3eyE/ft4iYhoOn3JeTEsfV88dz9fzxgKeH80BlM9uP1bG9tJ5dxxvYcrSOptPmdkZGGHKSY5mQFk9uT7KaGk98TCQ/emE/Te2d/N97FvGupSP7fzR1XBJPfOYCvvDIDr71r13sq2jk29fPG7U1jEXEd0drzqyyv3hS2ugHIoMqrfW8V+dNzWRrSR13/HULv/3QUjLH0PQwEV8MmpQaY+KAV4BY7+P/Ya39tjEmA3gUKACOAu+11tZ5n3MXcDvQDfyHtfaFgEQfxKpbXGSNsaG7AImxUczJTT7rvNKNR2ro7Lasmu5c715EhGHxuEjWHThJR1c3sVFD793pT6uri1cP1fChFfkBKeB0/cJcvvvv3Ty+7TjzJqT2rk+qpHRg/nofblk+mT9sOML/Pr+fi2eMO2Nd0x7Halt5cc8JPnXxNOJjfP+7umreeL75r108/VbFsJLS9s5ufvT8fuZPTOHGRRN55ZWDZ318VGQEcyekMHdCCh9c8XavZFN7JxUN7Ryvb6Oivp3y+jbKG9oor2/jrbJ6XtjV3ls0Y2pWIn+9fTmzx6cMOd7+pMRF8/uPLONHL+znt+sOsbeikZuX5LGsIJ2Z2ckDtrmIBF5FQxtfeGQHVU0dHKlu4dOF01gwMZX/99w+SmtbWeDwcmDSvwOVzQD877sXsv1YPV98dAe/LjrEt66b63BkIv7lS09pB3CptbbZGBMNbDDGPAfcDKy11t5jjPka8DXgq8aYucAtwDxgArDGGDPTWhtWiyzVNHeQ00/VyrFg6eR0/r61jK5ud7+9U+uLq4mNimBZQboD0b1tcXYkRWUdbDxcy0Uz/ZMgbyiuxtXl9vvQ3R5pCTFcOjubJ3eUc9c7ZrPpSC1ZSbFMyUoMyOvJ26IjI/jiFTP5/CM7+Pdb5dw4wFIHD7x2lAhj+PB5Qxt+mp4YwwXTPUN4v3r1rCEn03969SjlDe38+D2LRpS8JcdFkxwXzcyc5H7vd7stNS0uKhvbmZ6dNKzhumcTGWH42jtmMyc3mbuf2cs3/7XLG1cUSyansyw/naX56SyenDbmpjeIjGXr9p9k4xFPDYMLpmfykfMKGJ8ax7lTMnh5XxVTR3l5N/HNh1fm89c3SpiUkcCkjAR+ubaY43VtTocl4neDfiOwngoazd6b0d5/FrgRKPRufwAoAr7q3f6ItbYDOGKMOQgsB173Z+DBrrbFxZxc//Q+jLYl+ek88HoJ+0409dvjs764muVTMvz+ZXao5mZGEh8dyZq9lX5LStfsrSQ5LopzA9hzefOSPF7YXcn6g9VsOuJZn3S0l9UJV9cvnMDv1h3m/148wDvm554xtLS5o4tHNx/jmgW55KYOvfDUdQtz+a9/vMWbZQ1DGgpX2+Li1y8f5JJZ4zg/wMXDIiIM45Jjh1wwaqhuXDyRGxZNoLS2lS1H69hSUse2kjp+suYA1nqS17m5KSzNT2dZQTrL8jMYnzo2L+SJjAXH6lqJijCs+6/CUy44ZyXF8p5lkxyMTM7mezfN539umNd7OzMphpqW0V0uTGQ0+HSZ2hgTCWwFpgP3Wms3GmNyrLUVANbaCmNMT9fSROCNPk8v8247fZ93AHcA5OTkUFRUNOyD6Ku5udlv+xouay0nG9tpra0clVj8fcydbZ6hfY+s2cTl+aeuUVbb7uZgVRtL012Ot7OrrYU56VE8vb2US1JOjjixc1vL82+1MScjglfXv+KnKM8U6bYkRsPdT2zleL2bS3K7fW7LYPj7doI/j/vqCV38ZGsH331w7Rl/36tLOmnq6GJxfO2wXi+h0xJl4LfPbOSW2b4nfQ/u7aC5o4vLs94+zlB6rzOBqzI8/1o6EzhU301xnZvi+iYeeqPh7YJNcYZ3THLjucYpIv5UWtvGhLT4Ec3PF2f0HT2TlRTL7vJGB6MRCQyfklLv0NvFxpg04AljzPyzPLy/zOCMGtbW2vuA+wCWLVtmCwsLfQllUEVFRfhrX8PV2N5J1wsvcs6c6RReFPgKdv4+ZmstP9q+lsaYTAoLzznlvr9vOQa8xUevXuF4T3BRURG3XDSNr/zjLbJnLWHehJHNh9leWkfjC6/xgYsXUDiEAjfDcXPjLv76RgkAH7zS97YMhr9vJ/jzuC+2lldr3+D5Yy3cdcuFJMZ6Pgbdbsu3/6+IJZMT+NhNFwx7/0+Ub+at8kZ+fdHFPg3DPVrdwssvruOW5ZP44PULe7eHy3vd2e1mT3kjW0vq2FpSR1Z0bVgct8hoO1bbymStPTrmZSXFUt2snlIJPUO6XGatrcdzCftqoNIYkwvg/VnlfVgZ0HccSB5QPtJAx5LaMbpGaQ9jDEvz0/stdrS+uJqspFhmj+9/vtpou3R2NsbAmj1Vgz94EGv3VhEZYSicFfgCTjcv8SS9KXFRzBpg7p8EhjGGr75jNtXNHfzp1SO921/aV0VJTeuIq0pft3AC5Q3tbD82cLGwvv73hX3EREXwn5fPHNHrjlXRkREsmpTGxy6cwr0fXMLibM0z9QdjTKQxZrsx5mnv7QxjzGpjTLH3p7NFAWTUVDa289mHtrHjWD2TMrQe9liXmRhDU3sXHV1hVapFwsCgSakxZpy3hxRjTDxwObAPeAq41fuwW4Envb8/BdxijIk1xkwBZgCb/Bx3UOsZ6z8Wl4TpsTQ/g+P1bZxoaO/d5nZbXj1YzYXTM4NmDmRWUixLJqezZm/liPe1Zm8ly/LTSUsI/Pu2eFIas8cns2rmwFVgJXCWTE7nyrk5/G7dYepaPBeR7n/1CBNS47h63vgR7fuyOdnEREXw7zcrBn3s1pI6nt15gk+smkr2GC2MJkHr88DePre/hqc44Qxgrfe2hLiqxnZu/NWrPP2W5/NoOHPlJbhkeesB1Hg7QERChS89pbnAy8aYt4DNwGpr7dPAPcAVxphi4Arvbay1u4HHgD3A88Bnwq3ybrX3gyJrDK8htTTfcxF9W+nbvT17TzRS0+Ji1QznloLpz+Vzcth5vIGKhuFXoyura2XfiSYun5Pjx8gGZozh0U+ex4/evXDwB0tAfPmqWbS4uvh10UH2VjTy2qEaPnxewYjnWyXHRXPJrHE8u7MCt/uMmQu9rLXc/cwexiXHcscoDPOX8GGMyQOuBf7QZ/ONeIoS4v150yiHJQ54+q0KTjS286/PXMAfb13GrecVOB2SjFCmt8NDSamEmkG/fVlr37LWnmOtXWitnW+t/a53e4219jJr7Qzvz9o+z7nbWjvNWjvLWvtcIA8gGNWM8eG7AHNzU4iNijhlCO/64moALpwR2OqgQ3XFXE+NrbV7hz+E96V9nudeFqClYPqTGh+tJTEcNDMnmZuX5PHA6yX88Pl9xEVH8P7l/qlAed3CCVQ1dbD5aO2Aj3l+1wm2ldbzn5fP7J3XKuInPwO+Arj7bDulOCEweh924pj1xSeZkpXI4klpXDYnh9SE6MGfJEGtp6d0/cGTDkci4l/6JhQAtSEwfDcmKoJFeWmnJKUbiquZmZMUdOuvThuXREFmAmv2VvKhlUNbW7LHmr1VTM1K1DptYeY/r5jJUzvKKdp/kg+umOy3oduXzckmLjqCp9+qYMXUzDPud3W5+eHz+5iencR7l+X55TVFAIwx1wFV1tqtxpjCYTw/ZCvjBzu/V9J3W1492MqqiVEh1fbh/rfU5bbMy4zgf5/fj6kpYU5m/8vzhXs7+UJt5JvRaiclpQFQ3ewiOTaK2Chn1/EcqSX56fxxw2HaOz2jrzcdreXDw0z6AskYw+VzcvjL6yW0dHQNudepuaOLNw7VcOv5wXdsElgT0+L5yHn53P/qEW67oMBv+02IieKy2Tk8t6uCb18/94whwQ9tLOFoTSv3f3SZlmcQf7sAuMEYcw0QB6QYY/6Gtzihdwm3vsUJTxHKlfGDnb/b6LWD1bi6N/L+wsUUzh2dqSmjQX9LcOGqbq762Ss8eNDy76vPJ72fThC10+DURr4ZrXbSt6EAqGlxjemhuz2W5qfT2W3ZebyBTUdqcXW5g27obo/L5+bg6nazvnjow1k2FJ/E1e3mslGaTyrB5StXz+a5z1/E9Gz/VkG+bmEu1c0uNh05dQhvY3snP19bzHlTM7lklkZQin9Za++y1uZZawuAW4CXrLUfYuDihBKi1hWfJCrCsHLamaM1ZGyLi47kF7ecw4mGdn6//rDT4Yj4hZLSAKht6SBzDBc56rFkchrgqRC64WA1MZERrJiS4WxQA1iWn05qfDSrh7E0zOo9VaTGR7MsXyskhKOYqAhmBWCJo0tmZ5MYE8m/3zq1Cu9vig5R19rJ16+ZEzRVrCUs9FucUELXjtJ6FuSlkqQ56yFp0aQ0Zucms/N4g9OhiPiFktIAqGl2jen5pD0yk2KZkpXI1pI61hdXszQ/PWgL80RFRnDp7Gxe3l9F91kqnp6u2215eX8VhbPGaRil+FVcdCSXz83h+V0VdHZ76s2U17dx/4Yj3LR4AgvyUh2OUEKdtbbIWnud9/cBixNKaKptcZGdPPYvkMvA5oxPYW9Fo9NhiPiFvoUHQHWzi6wQGL4LnvUcXz9Uw96KxqAdutvj8jk51La42N5nGZvB7DhWR22LS0N3JSCuXZBLXWsnrx2qAeDHL+7H4lmORkQkkOpaO0PiArkMbE5uCtXNLsrqWp0ORWTElJT6mdttqWt1kZkYGlcnl+an09zRBcCqIE9KL5qZRXSkYfXeSp+fs2ZvFVERhotnBtfaqxIaLp41juTYKJ55q5xdxxt4Yvtxbju/gLz0BKdDE5EQZq2lvtXlt4riEpyWFXimHd3869eoamx3OBqRkVFS6mcNbZ10u23IXJ1c6p1nmZ4QzbwJwT3cMDkumpVTM1mzx/ekdO3eSs4tyCA1Xmu3if/FRkVyxbwcnt91gu8/s4fU+Gg+fcl0p8MSkRDX3NFFl9uSrnVJQ9rCvDT+fNu5NHd0ceO9r/KT1Qeob3U5HZbIsCgp9bMa7xqloVB9F2BGdhLpCdGsmjGOyIjgL8pyxdwcDp1s4fDJ5kEfe6y2lQOVzVw2RxVQJXCuXziBxvYu3jhcy+cunaELICIScPWtnQDqKQ0DhbOy+evtK5g6LpFfvVTMLfe94XRIIsOipNTPqps9V6iyQqD6LkBEhOHRT57Hf18/1+lQfNIzN3Tt3sGr8K7xDvO9IoTWb5Pgc8H0LFLjo8nPTAjKdX5FJPTUeXvL0pWUhoWl+ek8+PGV3HHRNA5UNmGt7wUfRYJFcJZSHcNqWzwnglDpKQWYmeP/5TICZWJaPHNzU1i9t5JPXDT1rI9du7eK6dlJ5GcmjlJ0Eo5ioiL4463LSImPJiZK1wFFJPDqvD2lGr4bXtITonFbaHF1Ox2KyJDpG5Kf1TR7hu+GypzSsejyuTlsOVpLXcvA8yoa2zvZeKRGQ3dlVCwryBhTF3dEZGzrmVeo4bvhJTnOcxGiqb3T4UhEhk5JqZ/1DN/N0InAMVfMycFt4eX9Aw/hfeXASTq7LZdrKRgREQkxPRdldYE8vCTHeQZANrV3ORyJyNApKfWzmpYO0hOiiYpU0zpl/sQUclJie+eM9mft3irSE6JZMjl9FCMTEREJvPo2T09ZSpxmaYWTt5NS9ZTK2KPMyc9qW1y6MukwYwyXz8lh3f6TdHSdOa+iq9vNy/uruGRW9pioKCwiIjIUjW1dJMVG6QJ5mOkZvtuonlIZg/Rp5WfVzS4yQ6Ty7lh2+dwcWlzdvHG49oz7tpXWU9/a2VupV0REJJQ0tneqlzQMpWj4roxhSkr9rKa5g6wQqrw7Vp03NZOEmEjW7DlzCO/avZVERxoumpnlQGQiIiKB1djWSYrWRA47KnQkY5mSUj+rbXGRmaieUqfFRUdy0YxxrNlbecZ6XWv2VrJiSmbvh7eIiEgoaWrvIkXnuLCjQkcylikp9aOubjd1rZ2aUxokLp+bQ0VDO7vLG3u3Ha1u4dDJFi0FIyIiIauxvZOUeA3fDTcJMZFERhj1lMqYpKTUj2q964Jp+G5wuGTWOIzhlCq8Pb9rKRgREQlVje2dGg0UhowxJMVGqadUxiQlpX5U412jVIWOgkNmUixLJ6efkpSu3VvFzJwkJmUkOBiZiIhI4DS2danQUZhKjouioU09pTL2KCn1o1otVh10Lp+bw67jjVQ0tNHQ2smmo7WquisiIiHL7bY0tavQUbiamZPM029VsK/2zCXxRIKZklI/qm7uADR8N5j0DNNds7eKogNVdLuthu6KiEjIanF14bao0FGY+un7FuO2lr01SkplbFFS6ke9w3dVfTdoTBuXyJSsRNbsqWTt3ioyE2NYPCnN6bBERET84rHNx1j5g7X8pugQnd1u7nvlMIAKHYWp1PhoxqfEUd1mB3+wSBDRJ5Yf1ba4iIwwpGrITNAwxnD5nGweeK2E2OgIrpo3nsgI43RYIiIiI7bvRCPf+NdOOrstf9xwmJf2VbL5aB0ACTH6iheu8tLjqW5ocDoMkSFRT6kf1bR0kJ4QQ4SSnqBy+ZwcXN1umtq7uFxLwYiISIj4bdEhYqMi+dyl06ludrH5aB13XDSVRXmpLMxLdTo8cUheeoJ6SmXM0WU0P6pudmk+aRBamp9OWkI0rR3drJoxzulwRERERqyqqZ1ndlbwwRX5vHtpHr986SB3vWM2n7x4mtOhicPy0uOp67B0dbuJilT/k4wNSkr9qLbFRaaS0qATFRnB7RdMob6tk8RY/cmLiEjwa3N1860nd1Fe38ZvP7z0lMJFRfur+OifNgPwkfPyyc9MZPf/XKVznAAwKSMBt4X/fmo3t51fQGpCNOOSYjFGI/kkeOnTy49qmjtYkJfmdBjSj89dNsPpEERERHz2j63H+MfWMgD++1+7+NF7FuHqcpMYG8Uv1hYDcNnsbKaOSwJQQiq9rl84gade38PftxzjoY2lAKTERXFn4XTuLFRPugQnfYL5UU2zi0ytUSoiIiIj8OjmUr715G7mTUjhirk5/GxNMc/uOgEWvnjlTHYdb+TmJRP53o3znQ5VglB8TCQfXxDLTz66kmffqsBt4ZXik/zw+X0U7a/itgumcPX88U6HKXIKJaV+0tHVTVNHl+aUioiIY4wxccArQCyec/w/rLXfNsZkAI8CBcBR4L3W2jqn4pSz++e24wB89erZnD8tk/rWTiIjDLuON3DPc/sAuGHRBPWOylllJ8fx0QumAHDr+QX86dUj/Pm1o3zqb1v5+S2LuXHxRIcjFHmbZj/7SW2Ld43SJK1RKiIijukALrXWLgIWA1cbY1YCXwPWWmtnAGu9tyVIHa9r453nTOSimeOIiozgOzfM41vXzeXr18wBYFxyLBdMz3I4ShlLIiMMH181lbVfupglk9P47r/3cKCyyemwRHopKfWTmmZPUpqh4bsiIuIQ69HsvRnt/WeBG4EHvNsfAG4a/ejEF13dbk40tjMxLf6M+xZNSuNPHz2X5z+/imhVVZVhiI2K5Ps3LcDV5eaan6/n/z27l0c2leJ2awkZcZbGffhJdXMHgIbvioiIo4wxkcBWYDpwr7V2ozEmx1pbAWCtrTDGaNHmIHWisZ1utyUv/cykFOCS2XrrZGTmTkhh3Vcu4RtP7OR3rxwGPBV71fsuTlJS6ie9w3cTNXxXREScY63tBhYbY9KAJ4wxPlfDMcbcAdwBkJOTQ1FRkV9iam5u9tu+QlVPG+2v7QagurSYotbDDkcVfPS35Btf2ul9eXBlVjz/WdTGP4q20VkWXh0r+lvyzWi1k5JSP+kdvqueUhERCQLW2npjTBFwNVBpjMn19pLmAlUDPOc+4D6AZcuW2cLCQr/EUlRUhL/2FaqKiorImbWEv7ywH2jn2sKVTMlKdDqsoKO/Jd8MpZ1+u/cVaiPiKCxcHtiggoz+lnwzWu2kCQl+Ut3SQUxkBMmqhCciIg4xxozz9pBijIkHLgf2AU8Bt3ofdivwpCMBylnd/ufNvLSvipsWTyA/I8HpcCRMLCtIZ+ORGl7aV6m5peIYJaV+UtPsIjMpBmOM06GIiEj4ygVeNsa8BWwGVltrnwbuAa4wxhQDV3hvSxDpclsqGtu57YICfnbLOURE6PuEjI7PXDKdxJgoPvbnLbzvvtdZvaeSzm6302FJmBk0KTXGTDLGvGyM2WuM2W2M+bx3e4YxZrUxptj7M73Pc+4yxhw0xuw3xlwVyAMIFrUtnqRURETEKdbat6y151hrF1pr51trv+vdXmOtvcxaO8P7s9bpWOVUjS6LtTAjO9npUCTMTEiL57kvrOLb18/lSHULn/jLFm6699XeIp4io8GXntIu4EvW2jnASuAzxpi5DLDmmfe+W4B5eOax/NpbCTCk1TR3kKEiRyIiIjIMde2eYZM5KfouIaMvOzmO2y6Ywut3XcbPb1nM7vJG/rm1zOmwJIwMmpRaayustdu8vzcBe4GJDLzm2Y3AI9baDmvtEeAgEPIzp6ubXWRpjVIREREZhreT0jiHI5FwFh0ZwY2LJzJtXCJvHK5xOhwJI0OqymOMKQDOATYCA615NhF4o8/TyrzbTt9XSJWdP9nURmtdpSOvHa4lrcPxuMPxmCE8jzscjxnC97hF6js8Sen4VCWl4ryVUzN5ckc5ba5u4mNCfsCjBAGfk1JjTBLwT+AL1trGsxT06e+OM0p5hVLZ+VZXF67nX2DR7OkUFk4b1deG8C1pHY7HHY7HDOF53OF4zBC+xy3hraGtk5JGN9GRhowEjboS5127MJeHNpVy6/2b+O2Hl5Kh0YASYD5V3zXGRONJSB+01j7u3VzpXeuM09Y8KwMm9Xl6HlDun3CDU88apSp0JCIiIr5wuy1vHqvnkU2lLPv+atYf72JSeoKq7kpQOH9aFr+45Rx2lNVz470bVPRIAm7QnlLj6RL9I7DXWvuTPnf1rHl2D6euefYU8JAx5ifABGAGsMmfQQebnv+oWUpKRUREZBDdbst//f1NHt9+HIC89HhW5XTz2ZtWOByZyNuuXzSB1PhoPnL/JjYUV3PTOWfMxhPxG1+G714AfBjYaYzZ4d32dTzJ6GPGmNuBUuA9ANba3caYx4A9eCr3fsZa2+3vwINJbYu3p1TVd0VERGQQ33t6D49vP84HV0zmqnnjObcgg42vrWdiWrzToYmc4tyCDADK6lodjkRC3aBJqbV2A/3PEwW4bIDn3A3cPYK4xpSe4bsaby8iIiL9OVjVxAu7K/nZmgN0dls+tHIy379pgdNhiZxVfEwkWUkxHK9vczoUCXFDqr4r/atu8Qzf1ZxSEREROZ21lo/9eQultZ7epuTYKL5w+UyHoxLxzcS0eMrqlJRKYCkp9YPaZhcJMZEkxKg5RURE5G3WWv7yegmlta2smJLBj9+ziJS4aFITop0OTcQneekJ7K1odDoMCXE+Vd+Vs6tpcWnoroiIiJyiq9vN1pI6vv3UbhJjIrnvw8uYlJGghFTGlInp8ZTVt9HtPmOFRxG/UVLqB9XNHWQmqciRiIiIeDy7s4Lp33iO368/DMCaL12sZFTGpMWT0nB1uXml+KTToUgIU1LqBzXNLrLUUyoiIiJej24+BsALuytZMSWD3FRV1pWx6fI5OWQlxfLjF/ZrvVIJGCWlflDb4lKRIxEREel1sKqZmMgIVs3I4uOrpjodjsiwxURF8P2b5rP/RBO/f+Ww0+FIiFJlnhGy1lLT0kGG1igVERERPGs6Hq9v4zvXz+WjF0xxOhyREbt6/njmTUhh5/EGp0OREKWe0hFqbO+is9uSpZ5SERERATYfrQVg+ZRMhyMR8Z+5E1LZXd6ItSp4JP6npHSEaltcgNYoFREREY9NR2pJiYti1vhkp0MR8Zu5E1JoaOtkj5aHkQBQUjpCNd4J35kavisiIiLAxiO1nFuQQWSEcToUEb9ZNT2L+OhI3nnva3z333t6O2ZE/EFJ6QhVN3v+Q2qdUhEREalu7uDwyRaWT8lwOhQRvyrISuTlLxfyznMm8ufXjnDx/77MI5tKnQ5LQoSS0hGqafH0lGZpnVIREZGwt+VoHQDLCpSUSugZnxrHD9+9kBe+cBFTxyVy9zN7nQ5JQoSS0hGqVU+piIiIeG05WktsVATzJ6Y4HYpIwMzISeaS2dk0dXTR1e12OhwJAUpKR6imxUVyXBQxUWpKERGRcLe5pI5Fk9KIjYp0OhSRgEqNjwY8K1GIjJQyqRGqbu7Q0F0RERGh1dXF7uMNnFuQ7nQoIgHXm5S2dTociYSCKKcDGOtqW1xkauiuiIhIWDpY1Ux5fRs7jtXz5rF6utxW80klLKTEeZLSBiWl4gdKSkeoptlFQVaC02GIiIjIKPvBs3u575XDABgDeenxXD4nmxWqvCthIDVBSan4j5LSEapp6WBJvobpiIiI84wxk4C/AOMBN3CftfbnxpgM4FGgADgKvNdaW+dUnKHA7bY8uvkYy/LT+eKVM5k3IbV3OKNIOHh7TqmSUhk5zSkdAbfbUtviIitJw3dFRCQodAFfstbOAVYCnzHGzAW+Bqy11s4A1npvyzBZa3lpXxUNbZ2899xJnD8tSwmphB0N3xV/Uk/pCNS3deK2aE6piIgEBWttBVDh/b3JGLMXmAjcCBR6H/YAUAR81YEQx7zq5g4+/sAWdhyrB2CpRktJmOq5EKOkVPxBSekI1DR3AJCh6rsiIhJkjDEFwDnARiDHm7Bira0wxmQP8Jw7gDsAcnJyKCoq8ksszc3NftuXU1zdlgf3unijogtr4ZzsSKIioHTXZo4ZM+L9h0IbjQa1k29Go52stUQZ2LX/MEWUBfS1AkF/S74ZrXZSUjoC1c0uALLUUyoiIkHEGJME/BP4grW20fiYNFlr7wPuA1i2bJktLCz0SzxFRUX4a19O+cfWMtaVvcnNSyZy+4VTmDch1a/7D4U2Gg1qJ9+MVjulvbqaxIxsCgsXBfy1/E1/S74ZrXbSnNIRqG3xJKWZ6ikVEZEgYYyJxpOQPmitfdy7udIYk+u9Pxeociq+seihjaV8+e9vkp+ZwP+9Z5HfE1KRsWpiegJ/31rGruMNTociY5yS0hGoafEM381UoSMREQkCxtMl+kdgr7X2J33uegq41fv7rcCTox3bWNXU3sn/e24vAHdePA1fe51FwsHdN80H4I3DNQ5HImOdhu+OQHWzC2MgPUFJqYiIBIULgA8DO40xO7zbvg7cAzxmjLkdKAXe40x4wctay6GTLVQ2ttPU3snkjER++Pw+Nhyspttt+eed56uokchp5uamYAw0qtiRjJCS0hGoae4gPSGGyAhdNRUREedZazcAA52ULhvNWMaaF3ZX8qm/bT1lW0xkBB9fNYWLZoxTQirSj4gIQ0pctCrwyogpKR2B2haXloMREREZ46y1/PWNowD85L2LmJyRwIHKZpZPyWB6dpKzwYkEuZT4KCWlMmJKSkegptlFhpJSERGRMWlDcTXP7arg5X1VlDe0819XzeLmJXkALCvIcDg6kbEhNV49pTJySkpHoLqlgznjU5wOQ0RERHz023WHeH7XCfZWNNLR5SYmMoJLZo/ji1fO4p3nTHQ6PJExR0mp+IOS0hGobXGp8q6IiMgYseNYPfc8t48FE1OZlJFAYmwUD358BUmx+jokMlyp8dFUNnY4HYaMcfoUHqbObjf1rZ1kJmqNUhERkbHgl2uLSUuI5uE7VpIYE4m1nkItIjJ86ikVf1BSOkx1LS4AMtRTKiIiEpRaOrrYeKSGrKRYHtpYytp9VXz5ypm9PaNaclRk5FR9V/xBSekwVTd7ktIsFToSEREJOu2d3Vzzi/WU1LQCEBlhWDUji4+cX+BsYCIhJiU+GleXm/bObuKiI50OR8YoJaXDVOvtKc1M0vBdERGRYFNc2UxJTStfvXo2E9LimDchhenZyU6HJRJyUuOjAXjlwEmunDfe4WhkrFJSOkw1LZ4J3VoSRkREJPjsO9EIwFXzcpg6TmuNigTKuQUZJMVG8dV/vsWls7OJioxwOiQZg/RXM0y9w3c1p1RERCToHKhsIiYqgvzMRKdDEQlps8Yn8/9uXkBdaydvltU7HY6MUUpKh6m2pYOoCENKXLTToYiIiEgf+0808cDrJczITiJS1XVFAu6iGeOIMPDMWyecDkXGKCWlw1TT7CIjMUal5EVERIJIRUMb7//9G0QYeNeSPKfDEQkLqQnR3LR4In9+7Qg/emEf1lqnQ5IxZtCk1BhzvzGmyhizq8+2DGPMamNMsfdnep/77jLGHDTG7DfGXBWowJ1W7U1KRUREJHg8+EYp9a0unv7cKj524RSnwxEJG99/53zOmZzOvS8f4nB1i9PhyBjjS0/pn4GrT9v2NWCttXYGsNZ7G2PMXOAWYJ73Ob82xoRkbeialg6yVHlXREQkKDS0dfKT1Qf41csHWTVjHNOzVdxIZDQlxETxnevnAZ4h9CJDMWhSaq19Bag9bfONwAPe3x8Abuqz/RFrbYe19ghwEFjun1CDS22Li0wVORIREXGUq8vNjmP13Hr/Jn75UjHL8tP5ytWznA5LJCzNyEnCGCWlMnTDXRImx1pbAWCtrTDGZHu3TwTe6PO4Mu+2kFOj4bsiIiKOOl7fxp1/28pbZQ1ERhh++f5zuG7hBKfDEglbcdGRFGQm8lZZPW63Ve0V8Zm/1ynt7y+v35nOxpg7gDsAcnJyKCoq8ksAzc3NftvXQFzdluaOLhqrjlNUdDKgr+WL0TjmYBSOxx2OxwzhedzheMwQvsctQ1Pb4uJzD2/j1YM1AHzh8hl8YPlkslPiHI5MRFZOzeThTaVc+n9FPPbJ8/T/Unwy3KS00hiT6+0lzQWqvNvLgEl9HpcHlPe3A2vtfcB9AMuWLbOFhYXDDOVURUVF+GtfAymvb4PVL3HugtkULp8c0NfyxWgcczAKx+MOx2OG8DzucDxmCN/jFt/tOFbPfz66g/L6Nv7rqlmcMzmN86dlOR2WiHh954a5zJ+Ywjee2MWGg9XcrCrY4oPhLgnzFHCr9/dbgSf7bL/FGBNrjJkCzAA2jSzE4FPT7AIgU4WORERERs2Wo7Xc/OtXaWrv5MGPr+Azl0xXQioSZGKjInuXYyqra3M4GhkrBu0pNcY8DBQCWcaYMuDbwD3AY8aY24FS4D0A1trdxpjHgD1AF/AZa213gGJ3THVLB4DmlIqIiIwSt9vyzX/tIjc1nmc/v4rU+GinQxKRAcRFR5KdHEtZXavTocgYMWhSaq19/wB3XTbA4+8G7h5JUMGu1ttTmqXquyIiIgHX0NrJfesPse9EEz9732IlpCJjQF56vHpKxWf+LnQUFmq8PaUavisiIhJYv1hbzC9fKqaz2zJvQgrXLcx1OiQR8UFeegI7jtU7HYaMEUpKh6Gm2UVMVASJMZFOhyIiIhKyOrvd/G7dIc4tyOC/rprFgompREUOtxyGiIymvPR4nnqznPf97nU6u91cv2gCt55XoGVipF9KSoehutlFVmIMxug/lYiISKC8VdZAi6ubD6/M55zJ6U6HIyJD8K6leew/0cTafZ5FOraV1rPxcC3/995FJMYqBZFT6S9iGGpbOjR0V0REgpIx5n7gOqDKWjvfuy0DeBQoAI4C77XW1jkVoy8qG9v5xF+2AJ51D0VkbJk2Lok/fvRcdh1vID8zgUc3H+MHz+7l5l+3cNsFBVyzMJeUOM0PFw+NgRmGmhYXmSpyJCIiwenPwNWnbfsasNZaOwNY670d1B7dfIzaFhffuGYO6ap2LzJmzZ+YSnJcNB9fNZU/3bacts5uvvb4Tt7xs/XsOt7gdHgSJJSUDkNNs0vLwYiISFCy1r4C1J62+UbgAe/vDwA3jWZMQ1Xf6uLJHcc5tyCdT1w01elwRMRPLp45jnX/VchjnzwPt7Vc98sNXPPz9fz1jRK63dbp8MRBSkqHyFpLTUsHWRq+KyIiY0eOtbYCwPsz2+F4BvTyvipW/fBlDle38KGV+U6HIyJ+Zoxh+ZQMnvrshXzxipkYA9/61y5+v/6w06GJgzSndIhaXd20d7rJVE+piIiEGGPMHcAdADk5ORQVFfllv83NzYPuq7HDUlTWydOHO8mON3x1WTyp9cUUFRX7JYZg50sbidrJV2OlnRZGwoIFlv9zRfLz1fsoO3qYi/KiiIkMfDHRsdJGThutdlJSOkQ1zS4ADd8VEZGxpNIYk2utrTDG5AJV/T3IWnsfcB/AsmXLbGFhoV9evKioiMH29aE/bGTDwWqmjUvkl+9fwtwJKX557bHClzYStZOvxlo75c9v5ouPvcnf9tZzsCOJ/3vvYiamxQf0NcdaGzlltNpJw3eHqLqlA0DDd0VEZCx5CrjV+/utwJMOxnKGqqZ2XjtUzZ2F01j7pcKwS0hFwt3UcUn86zMX8IN3LuCNw7Vc8ZN1tHd2Ox2WjCIlpUNU6+0pVfVdEREJRsaYh4HXgVnGmDJjzO3APcAVxphi4Arv7aBQVtfKj57fj9vCTYsnOh2OiDjoAysm89WrZ9Pq6mbfiSanw5FRpOG7Q1Tj7SnVOqUiIhKMrLXvH+Cuy0Y1EB/84Nm93PeKp7jJJy+eyqzxyQ5HJCJOu25hLj98fh+7yxtYPCnN6XBklCgpHaLqnp5SzSkVEREZtgOVTfxh/WGuXzSB/7h0OjNylJCKCOSlx5MSF8WmI7W8d9kkoiM1sDMc6F0eotoWF4kxkcRFRzodioiIyJhjreV4fRvfe3oPibFRfPeGeUpIRaSXMYal+ek8uaOcJd9bzc/WHMBarWEa6tRTOkQ1zR0auisiIjIMm4/W8quXDrLuwEkAvnntHNI18khETnPvB5fwyoFqnthexs/WFPPG4Ro+efE0CmeOw5jALxcjo09J6RDVtLi0HIyIiMgQHKjr5g/eJV9ioiL40hUzWTVzHIvyUp0OTUSCUEJMFFfPH89V83L45UsHeWhjKbf9aTOTMxK46x2zeceCXKdDFD9TUjpE1c0uJqbFOR2GiIhI0Gvp6OI/Ht5O0f520hPdfP6yGXx81RSS46KdDk1ExgBjDP9x2Qw+efFUnn6zgj9sOMIXHt3BjJwkpmdr2H8o0ZzSIapt6SAzUcN3RUREBpMQE0ljeycXTIzi+S9cxH9eMVMJqYgMWWxUJO9amscDt51LYmwUn31oOzuO1WuuaQhRT+kQWGupaXZpjVIREREfGGP4+6fOp6ioiCzVYxCREcpOieMn713Epx/cxk33vkpeejwrp2aycmom75g/nsRYpTZjld65IWhs66LLbTWnVERERETEAYWzstn49cv495sVvHLgJGv3VvKPrWUcqGzi69fMcTo8GSYlpUNQ09IBoKu9IiIiIiIOSY6L5gMrJvOBFZNxuy3v+u1rbCupczosGQHNKR2CmhYXgIbvioiIiIgEgYgIwzmT0tlV3kBXt9vpcGSYlJQOQU2zp6dUw3dFRERERILDokmptHe6+daTuyipaen3MdtL67jul+v5zIPb+Peb5bR2qkhSMNHwXR+1ubq575XDxEZFkJeW4HQ4IiIiIiICXDg9i2X56Tyy+RgPbzrG/IkpXDo7h/OmZjJvYgopcdH8fWsZByqbqW5y8czOCgwwb996ZmYns2hSGu9emqdCSQ5Sy/ugq9vN5x7ezvZj9fzmg0tJTVA5exERERGRYJCZFMs/7jyf8vo2ntxRztq9lfzqpWJ+sbYYgCWT09hWWs+Vc3P4zYeWsr74JP9ct4P6yBheP1zD49uP8/O1xdx+4RTOLchg/sQUEmKUJo0mtfYgrLX891O7WbO3ku/eOI+r5493OiQRERERETnNhLR47iycxp2F06hrcfFmWT27jjfwu3WHAbh8bg6REYbCWdlQEUNh4QoAtpXW8bM1xfzohf0AxEZFUJCZyOTMBBblpbKsIIMVUzIwxjh2bKFOSekg7n35IA9tLOXOwml85LwCp8MREREREZFBpCfGUDgrm8JZ2dx6fgGlta3MHp/S72OXTE7nLx9bTlldKwcqm3j1YA0lNa0cOtnM6j2VAMzITuKimeO4cHoWy6dkaKivn6k1z+LvW47x4xcPcPM5E/nKVbOcDkdERERERIYoOS6aeRNSB31cXnoCeekJXDo7p3dbQ1snz++q4Kk3y/nrGyX8ccMRoiIMiyalMTMnickZieRnJpCbGkduajzpidFYC3HRkYE8pJCjpHQARfur+NrjO1k1I4t73rVQ3fUiIiIiImEmNT6a9507mfedO5n2zm62ltSx4WA1m47U8uLuyt4lI0+XHBdFdnIs2clxZKfE9v6+IC+VJZPTiYnSIih9KSntx86yBj794DZm5STzmw8t1R+NiIiIiEiYi4uO5ILpWVwwPat3W1N7J6W1rZxoaKeioZ36VhfGGE42dVDV1E5lYwfbSuuoauygo8uzjmpUhCEvPZ7JmYnMzE5iZk4ykzISWDk1fOetKik9TWlNK7f9eRPpCTH8+bZzSdJ4cRERERER6UfP0ODBhgdba6lv7WTT0Vp2HKuntKaVktoW/vJGDS5vsvqfl89kxdQMclLiyE2NC6shwMq4+qhp7uDWP22iy2155GPLyU6JczokEREREREZ44wxpCfGcNW88Vw17+3VPNo7uznZ1MHXn9jJT9ccOOU5GYkxTM5IYGpWIgXef5MzEhifEse45FgiI0KnV1VJqVebq5vbH9hCeX0bD31iBdOzk5wOSUREREREQlhcdCSTMhL400fP5UBlM7UtLiob26loaON4fTslNS29a6n2FRlhyEyMISsplsykGNISYkhPiO79mZ4QQ6r3Z8/2lLiooB0erKQU6Op287mHt/FWWT2/+dBSluZnOB2SiIiIiIiEiajICOZO6H/JGvB0oJXUtnC8ro2KhnZONLRzsqmD6uYOalpcHKttpa61k8b2Tqztfx+REYa0+GjS+iSvp/6MYdb4JEdyobBPSq21fOvJXazZW8X3bpp/Sne6iIiIiIiI0+JjIpk9PmXAtVZ7dLstDW2d1LW6qG91Ud/aSV1rJ/WtLupaXb2/17d2cry+nd3ljdS1umjv9MxrjYwwbP3m5aQlxIzGYfUK+6T0ly8d5OFNx/jMJdP48Mp8p8MREREREREZlsgIQ0ZiDBmJQ0sq2zu7efVgNbc/sIU3Dtdy9fzR7agLqaR0zZ5Ktp/oomHHcTq63HR2u3H1+enqtqfcbmjr5Kk3y7l5yUS+fOUsp8MXEREREREZdXHRkayaMY746EgeeO0oJxraSIqLpqSyi2m1rUzKSAjo64dUUvqpv22ly21hx44BHxMTGUF0pCEmKoLoyAhuXDyBe25eGLSTfkVERPzFGHM18HMgEviDtfYeh0MSEZEgERMVwTULcvnntjJeP1zTuz0zr5KPXjAloK8dsKTUiRPfvz5zAdu3beX8lcuJiYwgJirCk4T2/Iw0Sj5FRCQsGWMigXuBK4AyYLMx5ilr7R5nIxMRkWDxf+9dxA9unk9LRzfN7V0Uvfo671g4IeCvG5Ck1KkT3/yJqVQXRzBtnJZzEREROc1y4KC19jCAMeYR4EZASamIiPSKjYokNirSs05qSiTjkmMD/poRAdpv74nPWusCek58IiIi4oyJwLE+t8u820RERBwVqOG7/Z34VgTotURERGRw/c1fOWU1O2PMHcAdADk5ORQVFfnlhZubm/22r1ClNvKN2sk3aqfBqY18M1rtFKikVCe+URSOxwzhedzheMwQnscdjscM4Xvco6QMmNTndh5Q3vcB1tr7gPsAli1bZgsLC/3ywkVFRfhrX6FKbeQbtZNv1E6DUxv5ZrTaKVBJqU58oygcjxnC87jD8ZghPI87HI8Zwve4R8lmYIYxZgpwHLgF+ICzIYmIiARuTmnvic8YE4PnxPdUgF5LREREBmGt7QI+C7wA7AUes9budjYqERGRAPWUWmu7jDE9J75I4H6d+ERERJxlrX0WeNbpOERERPoK2DqlOvGJiIiIiIjIYAI1fFdERERERERkUEpKRURERERExDFKSkVERERERMQxxlo7+KMCHYQxJ4ESP+0uC6j2077GinA8ZgjP4w7HY4bwPO5wPGbw33HnW2vH+WE/YUvn5lGnNvKN2sk3aqfBqY184892GvDcHBRJqT8ZY7ZYa5c5HcdoCsdjhvA87nA8ZgjP4w7HY4bwPe5Qp/d1cGoj36idfKN2GpzayDej1U4avisiIiIiIiKOUVIqIiIiIiIijgnFpPQ+pwNwQDgeM4TncYfjMUN4Hnc4HjOE73GHOr2vg1Mb+Ubt5Bu10+DURr4ZlXYKuTmlIiIiIiIiMnaEYk+piIiIiIiIjBEhk5QaY642xuw3xhw0xnzN6XhGizHmqDFmpzFmhzFmi9PxBIIx5n5jTJUxZlefbRnGmNXGmGLvz3QnYwyEAY77O8aY4973e4cx5honY/Q3Y8wkY8zLxpi9xpjdxpjPe7eH9Pt9luMO2ffbGBNnjNlkjHnTe8z/490e0u91uAnXc3N/hnouM8bc5W23/caYq5yJenQN5xwQpu005M/PcGwnAGNMpDFmuzHmae9ttdFp+sslnGinkEhKjTGRwL3AO4C5wPuNMXOdjWpUXWKtXRzCZa3/DFx92ravAWuttTOAtd7boebPnHncAD/1vt+LrbXPjnJMgdYFfMlaOwdYCXzG+3851N/vgY4bQvf97gAutdYuAhYDVxtjVhL673XY0Ln5DH/Gx3OZt51uAeZ5n/Nrb3uGuiGdA8K4nYb0+RnG7QTweWBvn9tqo/6dnkuMejuFRFIKLAcOWmsPW2tdwCPAjQ7HJH5irX0FqD1t843AA97fHwBuGs2YRsMAxx3SrLUV1tpt3t+b8JxIJhLi7/dZjjtkWY9m781o7z9LiL/XYUbn5j6GeC67EXjEWtthrT0CHMTTniFtGOeAcG2noX5+hmU7GWPygGuBP/TZrDbyzai3U6gkpROBY31ulxHiX+j6sMCLxpitxpg7nA5mFOVYayvAcxIDsh2OZzR91hjzlncoWMgObTTGFADnABsJo/f7tOOGEH6/vcOqdgBVwGprbVi912EgnM/Nvhro7z3s287Hc0DYttMQPz/DtZ1+BnwFcPfZpjY6U3+5xKi3U6gkpaafbeFSVvgCa+0SPMOjPmOMucjpgCSgfgNMwzNcpwL4P0ejCRBjTBLwT+AL1tpGp+MZLf0cd0i/39babmvtYiAPWG6Mme9wSOJf4XxuHqmwbrshnAPCtp2G+PkZdu1kjLkOqLLWbvX1Kf1sC+k26mMouUTA2ilUktIyYFKf23lAuUOxjCprbbn3ZxXwBOEz1KDSGJML4P1Z5XA8o8JaW+k9EbmB3xOC77cxJhrPl5EHrbWPezeH/Pvd33GHw/sNYK2tB4rwzE8J+fc6jITtuXkIBvp7D9u2G+I5IGzbqYePn5/h2E4XADcYY47imTpwqTHmb6iNzjBALjHq7RQqSelmYIYxZooxJgbPBNynHI4p4IwxicaY5J7fgSuBXWd/Vsh4CrjV+/utwJMOxjJqej4gvN5JiL3fxhgD/BHYa639SZ+7Qvr9Hui4Q/n9NsaMM8akeX+PBy4H9hHi73WYCctz8xAN9Pf+FHCLMSbWGDMFmAFsciC+UTWMc0C4ttNQPz/Drp2stXdZa/OstQV4PntestZ+CLXRKc6SS4x6O0X5YydOs9Z2GWM+C7wARAL3W2t3OxzWaMgBnvB8hhMFPGStfd7ZkPzPGPMwUAhkGWPKgG8D9wCPGWNuB0qB9zgXYWAMcNyFxpjFeIZKHAU+6VR8AXIB8GFgp3euDMDXCf33e6Djfn8Iv9+5wAPeqn0RwGPW2qeNMa8T2u912Ajjc3O/hnIus9buNsY8BuzBU5H2M9babkcCH11DOgeEcTsN6fMzjNupP/pbOlW/uYQxZjOj3E7G2nAZLi0iIiIiIiLBJlSG74qIiIiIiMgYpKRUREREREREHKOkVERERERERByjpFREREREREQco6RUREREREREHKOkVERERERERByjpFREREREREQco6RUREREREREHPP/AXjKWxn4Vk4eAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████| 30001/30001 [03:13<00:00, 155.24it/s]\n"
     ]
    }
   ],
   "source": [
    "train_agent(env, agent, target_network, optimizer, td_loss_fn=td_loss_dqn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "final score: 264.73333333333335\n"
     ]
    }
   ],
   "source": [
    "final_score = evaluate(\n",
    "  make_env(ENV_NAME),\n",
    "  agent, n_games=30, greedy=True, t_max=1000\n",
    ")\n",
    "print('final score:', final_score)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let us record a video of trained agent**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_animation(env, agent, save_dir):\n",
    "    \n",
    "    try:\n",
    "        env = gym.wrappers.Monitor(\n",
    "            env, save_dir, video_callable=lambda id: True, force=True, mode='evaluation')\n",
    "    except gym.error.Error as e:\n",
    "        print(e)\n",
    "\n",
    "    if not os.path.exists(save_dir):\n",
    "        os.makedirs(save_dir)\n",
    "        \n",
    "    state = env.reset()\n",
    "    reward = 0\n",
    "    while True:\n",
    "        qvalues = agent.get_qvalues([state])\n",
    "        action = qvalues.argmax(axis=-1)[0]\n",
    "        state, r, done, _ = env.step(action)\n",
    "        reward += r\n",
    "        if done:\n",
    "            print('Got reward: {}'.format(reward))\n",
    "            break\n",
    "            \n",
    "def display_animation(filepath):\n",
    "    video = io.open(filepath, 'r+b').read()\n",
    "    encoded = base64.b64encode(video)\n",
    "    return HTML(data='''<video alt=\"test\" controls>\n",
    "                <source src=\"data:video/mp4;base64,{0}\" type=\"video/mp4\" />\n",
    "                 </video>'''.format(encoded.decode('ascii')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Got reward: 281.0\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<video alt=\"test\" controls>\n",
       "                <source src=\"data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAAIZnJlZQAATzFtZGF0AAACoQYF//+d3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3JlIDE2MSAtIEguMjY0L01QRUctNCBBVkMgY29kZWMgLSBDb3B5bGVmdCAyMDAzLTIwMjAgLSBodHRwOi8vd3d3LnZpZGVvbGFuLm9yZy94MjY0Lmh0bWwgLSBvcHRpb25zOiBjYWJhYz0xIHJlZj0zIGRlYmxvY2s9MTowOjAgYW5hbHlzZT0weDM6MHgxMTMgbWU9aGV4IHN1Ym1lPTcgcHN5PTEgcHN5X3JkPTEuMDA6MC4wMCBtaXhlZF9yZWY9MSBtZV9yYW5nZT0xNiBjaHJvbWFfbWU9MSB0cmVsbGlzPTEgOHg4ZGN0PTEgY3FtPTAgZGVhZHpvbmU9MjEsMTEgZmFzdF9wc2tpcD0xIGNocm9tYV9xcF9vZmZzZXQ9LTIgdGhyZWFkcz0xMiBsb29rYWhlYWRfdGhyZWFkcz0yIHNsaWNlZF90aHJlYWRzPTAgbnI9MCBkZWNpbWF0ZT0xIGludGVybGFjZWQ9MCBibHVyYXlfY29tcGF0PTAgY29uc3RyYWluZWRfaW50cmE9MCBiZnJhbWVzPTMgYl9weXJhbWlkPTIgYl9hZGFwdD0xIGJfYmlhcz0wIGRpcmVjdD0xIHdlaWdodGI9MSBvcGVuX2dvcD0wIHdlaWdodHA9MiBrZXlpbnQ9MjUwIGtleWludF9taW49MjUgc2NlbmVjdXQ9NDAgaW50cmFfcmVmcmVzaD0wIHJjX2xvb2thaGVhZD00MCByYz1jcmYgbWJ0cmVlPTEgY3JmPTIzLjAgcWNvbXA9MC42MCBxcG1pbj0wIHFwbWF4PTY5IHFwc3RlcD00IGlwX3JhdGlvPTEuNDAgYXE9MToxLjAwAIAAAAIfZYiEACf//vWxfApqyfOKDOgyLuGXJMmutiLibQDAFR8gAAADAAA7Zb/FzICUal8AAAMAQkANQH0GUHmImKsVAlgxwKOLkKiQ6CFQi9tPXsvrVcjr0MOnelCN+qCK7dapMf/Tz8YDg6TdK2YSzz2t3g5896DZOrLbKJ2RjhSaAeqHGCGVZYpEWC1XsF678AjvoJlHYfPyvH2Xh3u1NacWt7UhqRHeXrB9c2CiBl8ClsAKIAWnHRYzRimZorPyezO4D6AdIRFZjO0KHI316K3yKJ64TFP4NCLdpmS64uHqKuiYPgTQx9OQZPp77rD+mvjm98ICbE8oFmVfUG20kkZ/5H8WO0LqekDlzag/YK3+Cl6nPZetfT5SaJOlssOsw7GPEI1PJ/XGhZFom0LEMccT9owIc7w6DZRmxR1yqsruKTmgJEW0UAAfLTsXipTtzAI1EKTH7hsyjBdYvTEzor/WcTUbXGWxa2N2Ou52J1cEf2be5DSU6AHNkTz+ME6+pC64CDzoQLPiuxVQaiZoRCqzvwJlE1kM/nckTc94ZobHP6Q/sg11tUzs9q9fVN9DIrOzphyjJ9k9NbPQLdUUK/fCFdlDZu/t2PeCpnavT0Myc2ZT44ASN8DDpNzz6Cc86kp+XSkvghqgUp9u2El4GTv75sdiJiZNE5+P+Lpn4yh6oMIr45PVOMcqzJawAB9gBTxAAAADAAADAAADAAADAO6BAAAAh0GaJGxCv/44QAABDRvSoAQjurrOhji+v1B7UkHmHIcvf6JzR+dvoPJKRkPSwCJqWkFOT+4GuEmgNii7eEalOqrcz9gwdOeBJllDEupRvGf018sPt5d0EFMg072Y1PiE180BzAFT3bhcCuqTQys/p5Z+fkKw9OKOzc6cYODEgxQZrWRTlXksYAAAAEdBnkJ4hH8AABa+Ww+rBYzeBnma9CGSUtI/rnMiYzRhQJSA3/R2KhWhfO4Cs37BKgnuXv2PIcQpU6sKgiugOAkshYfy80EPFwAAACQBnmF0R/8AAA1/UMaWVt41MIOWYxdkWxwC340OcXUS0yR8Qv8AAAA6AZ5jakf/AAAjkizIG0DmA8SX2WWsne4ruEzyn6XPiVABMetO1aEKaGz/f7vFTFb5YeaVuqNdWjUW0QAAAJZBmmZJqEFomUwU8J/98QAAAwKgxg6lduAmtvwL66hDy6Lm3Ke81EuEDRZrpuz6S4yv4e4R5BBiCez+JPkvQSG6wrMXy99M7sX58LL4htW/XZwzCbHlgiH4L1D6/vd5tbk1v6ysgxaZUaxc4FixTfmx2N0bXcwju05SkxbFE0zVbeGOoqGzS2f8YClmrGKUl/oG+o3s/yEAAAAiAZ6Fakf/AAAjvx01HmzyU/WQDvMsT1ZBgbXYBmGx6tS2gQAAAGVBmopJ4QpSZTAhX/44QAABDYQYsABtVQavjT8XP5GGpQwuQXo3Fpb+09IXSG164zj49l3ng1eN/OTSk7mK+I0zYH7loGyfo3Q0GAiGinpPhnLPYOYNtrrd4kTkSCQDtVFU/RnAMQAAAC9BnqhFNEwj/wAAAwM5Ma/JwAAe3P/IG2e0qJS+5oYhpZcmjlqegTM7Zms1U+IOCAAAACsBnsd0R/8AAAMB5rGxFe7sRipi5Y7xRYXn4gAmPQvdG35C82U9jpZqgtSAAAAAGwGeyWpH/wAABRphDw2ixuJ5CMXuzjf9b9optQAAADxBms5JqEFomUwIT//98QAAAwKf7v8hATNkh3OQ8VevvG8Y74Jju1+AMYPD/AzkpXfPemtMd3st56gRVnAAAAAmQZ7sRREsI/8AABa8TjLmgAkfjIcr72nM5gkLXhxBgNphWqPNSsAAAAAnAZ8LdEf/AAAkw0aAgcBFvKo7cPKM2ZkL/Pjb1iIAJj0Y4vMAmiJ3AAAAHwGfDWpH/wAAJMe3p9x6Mn2jK2i+LpnLdF70ztVBtYEAAAAzQZsSSahBbJlMCF///oywAAAamTpPGq/SShoALG+6a9V2/8b7AaM9XyJkUcgn1ZCg0DPhAAAAJEGfMEUVLCP/AAAWvnW0yAfx06eVTvhCKeJEzyQ2/U99ojgiRgAAACABn090R/8AACPDQKtRThAgBKpQcvbZmSaJsEMwLUGrYAAAABQBn1FqR/8AACO/Gs570zS1g9iJgQAAADNBm1ZJqEFsmUwIV//+OEAAAQ2SsEZc+hLYwGNxWThyYSve4TPhzjcO7554LSIsu7z8GFAAAAAwQZ90RRUsI/8AABZsGyGr15JAAcQDgLXIMbaBDutsdisdcjWI3hLEzbebqbsJE3g8AAAAFAGfk3RH/wAAI8CanLmcVGY/4WpvAAAAEgGflWpH/wAAI78b2Pq/V5LlQAAAACRBm5pJqEFsmUwIT//98QAAAwKf7v8hANGPuPgDrRf57bmAA2cAAAATQZ+4RRUsI/8AABbAn5AyXU7EHQAAABEBn9d0R/8AACO4wex672s2oAAAABEBn9lqR/8AACO/G9j1svxBwQAAAGVBm95JqEFsmUwIX//+jLAAAEgQ9Cshz8AJ5xzPNlGUPu04fc4h0lCIWnL/45loHi2Mdi5rn1Ul225kHBCWLqe6UMGIg2ttG7acYBYJZ6Sw8HP1w5nMQfwAafQG0ut2MeEk5t+SQAAAADFBn/xFFSwj/wAAFweUoHdkrwwrhUhQA2zyMYEC2KbApySyZCCs1GOLgRSVt4tTVbUhAAAAMQGeG3RH/wAAJKt0h2xYAONfBGOmLnV6yxFsEZnB8zxF1jqfPF4ahVWxeORzYT7takEAAAAXAZ4dakf/AAAkvwQnEsr+fFA+Alk3emAAAAAfQZoCSahBbJlMCFf//jhAAAADABzu+oA2s+ukY8TOaAAAABdBniBFFSwj/wAAF051o+kCDOdXGHLlQQAAABMBnl90R/8AACTDF3VRUD6w8JuAAAAAEwGeQWpH/wAAJL8EK8Byi9RHXkEAAAA4QZpGSahBbJlMCE///fEAAAMCsQLw4PSB7YdufJszLoVJWDjJ/dZ3F9gsV6jKUis0ALFcdJoVtmAAAAAYQZ5kRRUsI/8AABdQmz92a5YJgQEMKTf5AAAAEgGeg3RH/wAAJMMXdVFQ0STCbwAAABgBnoVqR/8AACS/BCcS0CVBBXTQAfsvFf8AAAA5QZqKSahBbJlMCF///oywAAAbT+WNAxgCZ9bTFq+ij5eBgwg7+x6Bm5TEeNgVDw3crGjvXWS5oFlBAAAASkGeqEUVLCP/AAAXSTMKQiTKn4gAsECWISpYXj7KLbVWlnefKCrq1hTGS2mm9bWB8CJsB0pFM6fLmQTXrhQczXrkwI6psuijb8QWAAAALgGex3RH/wAAJMLMqUuAakH+gAcerQOmYDqlD/9LtRKOM8TJxVQtD34f954gDggAAAAaAZ7Jakf/AAAkvwQryE6D+R/ItrsBTJHs9akAAABFQZrOSahBbJlMCFf//jhAAABm5/cAChN80X+CCbFkvmJOJ47i7Yv//lgcNM6I7F/+/6ftkXQPqUl2Tvz8mru1YR10/MOqAAAAH0Ge7EUVLCP/AAADAC2BMc55rrelHqEkdACleCzP48AAAAApAZ8LdEf/AAADAEccHogA4LNvWzuOKsfO4aOfyUBbPWbAj8lcCn093uEAAAAWAZ8Nakf/AAADAAqGYSCa1yrQCxH/gQAAAHJBmxJJqEFsmUwIT//98QAAAwKxl3v7ci4L48An842dtOQRTz2XgY5nZT8XX467zrs1BCQORenoFtMZE13J6/MUm18Ohw9woijYU5vUiKhc8Jf9/W2E6J3jinNu/8NdN6qw2QWUYCP8TNJt6kuLflZR780AAAAhQZ8wRRUsI/8AABdMQ8+MRn83QWAOVomFpFEQABgPjnteAAAAKwGfT3RH/wAAJLrOtSADddC8PKLSdDZ80ZL5OUM7jT+sPp8AqBASpbaDvcAAAAAgAZ9Rakf/AAAkvYOByXR6cX0Mk+INMN4VXii7E1RNrUkAAAA+QZtWSahBbJlMCFf//jhAAAEU+r3DND0rwCaFwKKXijHels89Xoo0Fghz1BuaqaSdcTRc02PFlA9zZ6xglnAAAAAlQZ90RRUsI/8AABdFKzdrbU1vc6LTkbm8ts11rdRkWisSsfvfjwAAACABn5N0R/8AACSr63YghxiIU+taAZhHMKS7qTW1T012pQAAACoBn5VqR/8AAAVDNNWxqW37AAOCzb2WbgQ9nstkxwlScQn5t3VjNeT1614AAAAxQZuaSahBbJlMCE///fEAAAMCse3zN5zdCJkoy58p0ADYcoLGOfRqYu7YHdElcrAg4QAAADRBn7hFFSwj/wAAF03d2jauMQAIwGSiiaIUB6/uqzgSVK2X57sHrw4ulMQCm+mgK64036LhAAAAFwGf13RH/wAAJMLkSvgmtcZdTDgE2fjQAAAAFgGf2WpH/wAAJL8EJxXCe2U/RaXAV/kAAABtQZveSahBbJlMCFf//jhAAAEVKiNsBW6kWYaDws8ML01nRjt7kR5V66MUFlwEHrmvV/S5IhBruF7rcw2p2js8YVHAgaZM/fpwDdgggfDCNXOR0S0cboBh+jhJjoKH4+1tE0IEQ4FcOAlYvvyoHgAAAC5Bn/xFFSwj/wAAF0xDz75QusVA2AEXP9oDYMQkMhEfbOqk/cZA/dUOD79y8ifNAAAAIAGeG3RH/wAABUIyLMG3hGHQAt4gt9YfIBj5gPA9lTVNAAAAGQGeHWpH/wAAJL8EK8E5MncDd6xOMRVI7UgAAAA+QZoCSahBbJlMCE///fEAAAMCsb1pmipgYf+QAIfj5eEVBYdXmRWKkkSdTKG6k4hnq7tcuWrvbBWuVBoTLFwAAAAXQZ4gRRUsI/8AABdQmG6ZY/bQTDsIoTcAAAASAZ5fdEf/AAAkwxd1UVDRJMJuAAAAGAGeQWpH/wAAJLHSboAIwEkSIXYtgXADjwAAAERBmkZJqEFsmUwIT//98QAAAwD9k1d5EahyjBZPoITkW2cQ5fHoAkz1N+sLq1Wrtx1IZICNDmM+Wvz67KtXgZ2h0afT2gAAACtBnmRFFSwj/wAACKv65xvNLy6fYABuaRcv31IlvsUbVpx9qYp5vx+kN9HTAAAAHQGeg3RH/wAAI9hcwBEgkiRDKCh7RuNjkwTkkCphAAAAGQGehWpH/wAAAwC/WF+WCsFgAncUZ9u8kakAAABuQZqKSahBbJlMCFf//jhAAAEVowQ0owE8AIGTCzUeVZXfwL91SPXB7R2y/OD7RSXhS8r0ANfoRpNhwAvyzOxlfcyT1iK+x6znC9vGV0mFIsVFMg2cajOiRi2QkG78nlTJMKn3Cn3Llzq00akTmCMAAAAkQZ6oRRUsI/8AABdMQ823FoOTKAVq//7mqp7vVPCmP58BkM+PAAAAFwGex3RH/wAAAwAKh44FwnpuZP0GM5VgAAAALQGeyWpH/wAAJLnN3uIaIlC0ACWDjfpudRWjwSj6WEHghU2lb+ywIPb11DORYQAAACVBms5JqEFsmUwIT//98QAAAwKxvXw4FLlu5pKVDBq+XKGrSCoYAAAAHEGe7EUVLCP/AAAXRVpZSHX/RPp3eMAqWivSO1IAAAAZAZ8LdEf/AAAkrDxbF31sN+pOc6xx57f5tQAAABkBnw1qR/8AAAMAGulsI6EWDbGypxJcAsdfAAAAOkGbEkmoQWyZTAhP//3xAAADArLV4PClRU5S3MkQL24T9KiasxVM9hSQYfHtbWueWDA7vYYxgByjULEAAAAwQZ8wRRUsI/8AABdFKzdrXGwANHCsG4GzEaNO0+hcul1BlkuyeLrKPxKH++q57OcEAAAAIwGfT3RH/wAAI6xCVCT6B3oAC5/dvkQsgCDcUMeqdIqSAA8IAAAAHQGfUWpH/wAAAwC++2jDce0tkoCNyYf+Q3vh42pBAAAANEGbVkmoQWyZTAhX//44QAABFcjwAx5bB/VAK1Lhk4L6+k4XkV28aFH5duN0Rpj5+tTt8j4AAABGQZ90RRUsI/8AABa1X0fN0AJphAigWUyk+KVvQyIjPhVubjKLc0xfJo1iSvgLEuvJOYkp1a4HS8FuQOHqrXTrlJuwfxci4AAAABsBn5N0R/8AACTDF2ipxcspKosegCTW8d3hRYEAAAArAZ+Vakf/AAANz7aMNzfODamOSf8iUjWswQAAJzPyqQS//kz7sxkKlKQtLwAAAFlBm5pJqEFsmUwIT//98QAAAwKxAq02S0NIUAGe/zW4wr7giAyQxxZIiCkUX10eiHsmnoyY5693DRE7/zMpM9H62q60S07ZVJ6u+mtggtm66pX/9BIEMascwQAAAB9Bn7hFFSwj/wAAF0xDzbcVzLI0rQE6U6fZyBt21ScFAAAAHQGf13RH/wAAAwC+4A0hJqx+FkX9TIqVHzARoF3uAAAAEwGf2WpH/wAAJD3MDowfyG98ARcAAAA6QZveSahBbJlMCE///fEAAAMCs5bnMayOr5NiPiJzpI3gAWrCwx4ViibJIyA8M85odDnKhBpwFe6bgAAAACVBn/xFFSwj/wAAF0xDzbcWMADxSGAcf5pS4zj751s+xJ0oBG1JAAAALAGeG3RH/wAAJLf/SG6JsMFPg0R+bAAuJ/XL91Oh2RiYMEYN6MTXdommc/HBAAAAGgGeHWpH/wAAJL8EJxLO5vzRJl+NnYO792pAAAAAbUGaAkmoQWyZTAhP//3xAAADArG8jTACkPDQ5STX4/vbn7Qy+Bzhy9KXtUiECv+kBKuC+qWn6hFNrf00G2mdc5tEL5KUiuYFRntTp1ipy8CJ3kDV7q08oth6fAEgDSdl4D6O3D2XWR+Wd9HKXAgAAAA1QZ4gRRUsI/8AABdQVpPOXzL9Q1thvAIACcBW1MRKHIjCthfG7qw7EG1R9QZane5HJDy1+90AAAA3AZ5fdEf/AAAkw0JXukWwA6AFrMJfUvrlbi9eAdpcfWa30oGrHXJzIbRUYB8GeqRK1vU/FrRwQAAAABkBnkFqR/8AACS/HAg4vatTLxsyrxfjlE3BAAAAUEGaRkmoQWyZTAhP//3xAAADArG9cJJFmCymZypQqdTkCbQowgvz/x/hALQu2DucrocPgE7FsnsJ05QMuKn1gzsmaf5tGbpByC9WbSLCk5HMAAAAJEGeZEUVLCP/AAAXRVhVis4rqajRe+BvX4gmz1iYR3h68HeWpQAAADYBnoN0R/8AACSsOQSaR4FVnSACTKkAFO+WJzMriQ/LT+KnMyY+mZNZm++uQiOZ6Uvfh4ww2cEAAAAlAZ6Fakf/AAADAL79IuPpLT0BgAun8an0N4/V1PGTW0ujTEFe4QAAAExBmopJqEFsmUwIT//98QAAAwKxAvDgJko4zmPX3flr6HQaJpByOltCxp5Pn2q4VpekwZMZEMW2t9MKLhLltUmE1Qvo9MTeB2iWaJXhAAAAHUGeqEUVLCP/AAAXRVhVis4/+mn0ZLG5feeWnYfKAAAAHgGex3RH/wAAJKw5BUrAqAFrPp6SX74YCGB7BmwCpgAAACQBnslqR/8AAAMAR2Rbfg4ALhhCY77oe13f1bvJAk2Gk1OKLa8AAABCQZrOSahBbJlMCE///fEAAAMCsPcZugFJOp4iNoQCYI/+1VlnOecnTeV/Ibqxpwm3S27NFzOmga/PsSizHSX7c4yOAAAAIUGe7EUVLCP/AAAXS4rQhLlxnBp0NV3nEgWDkBtKYGjdwAAAABgBnwt0R/8AACS3/0FA/OY2CCyzMyDNllEAAAAZAZ8Nakf/AAAkvxvezzO9wN4pP9MceEfX7wAAAEtBmxJJqEFsmUwIT//98QAAAwKxAsSV/FX9XYKHDzACE0k25J8i5HzDgdebahdyHBOBtoqbl4kBINw9mAFf/Q0mNXT1rFQCuEOgnVUAAAAvQZ8wRRUsI/8AABdQVpIbh3DfW458iMOw+FiAD2673MLuy0jCefFh7wrLwA4K+mAAAAApAZ9PdEf/AAAkwzufqqSF8kakngALnplbv8kW7Mawqv1UDoYNYMFZ6YAAAAAqAZ9Rakf/AAAkni/iWEcHEe9uw6AE0uafFRUyV7mwDr2cfRsC4VzTwNiLAAAARkGbVkmoQWyZTAhP//3xAAADArMm8WrT7MKPmL2iAEqu8ZctL4c8FjZ5RcRxzB36i4Tr6EHcWmkguCrq7B/ZRMovyGHZq2AAAAA3QZ90RRUsI/8AABdDSI/4i5UALqFOXNpJXgrZXVy6zZCvBq7mTmv/yDfHNKjzpb1K3GuPopcO0AAAACYBn5N0R/8AACSsPmyEjsKPgALnPN8jV0neWjoGuYopT0Apr1KgQQAAACkBn5VqR/8AACTHK/w3knP8AEsfr/rDfhFC8/pm3lifp7TVsSiARUV3EAAAAE1Bm5pJqEFsmUwIT//98QAAAwLDAp7aunvHM1714ieDPXu4PRBbTD2QArlOlVZKnibTa9wCImT2MZgWWJRFNfEjNGYWMXme0xI2Qn9JqQAAADxBn7hFFSwj/wAAF+JPB3EBQACKukjvNlX8B+B3L1izDdxAwmTrspccVpvniUnyV0bXZ1Iw1iqJzsoZ04EAAAAeAZ/XdEf/AAAlwxdoqcbNdMmkJyGmD9lRixjQPUCAAAAAJAGf2WpH/wAAJL8fhmLilPle4Idi/MAJYpuAZwta0uL6lCDx6QAAAFhBm95JqEFsmUwIT//98QAAAwLC9trALNyjA/xABOIL2/tbmfldxBD0CrjxGWJX9nuBQvisaGzJiXXUAy1ajIz6vzjRC3tTtX0wGojGbgXkka1+GkbMdLglAAAAM0Gf/EUVLCP/AAAX7ZZ0xps6VuMfUPBnAC3i4yD/HkySNewgG5s3iSxdmSkCdyBw2c2OnQAAAC0Bnht0R/8AACWr+Ft4F1EVnmZ315/xogBGRLeyBAtE8JydZtQALQRjRaSI5BkAAAAlAZ4dakf/AAAOK/S2T5V/PibWE8AIqfpe0/uqk5wudcFa8d0HHAAAAFlBmgJJqEFsmUwIT//98QAAAwLDun3QBPUfl9bFASxOIArN89CvqYFdj4LGrsH1rdqd3pTs9uglpPOAHGghhf5tQluuc5sBnFQ11EeXIORob5HtiHAM9UFAdgAAAD5BniBFFSwj/wAAF+JrOdvhACZYlhJvORnA6RfT6FUYWHxyTDTd+KTRmiEzxxUn3Zo6WYFhJnslHs3J+LYO6wAAACoBnl90R/8AACXDF2iprLi+XsADjaXzsyXj+IWwOoF/vwT72M++n+miDtAAAAAuAZ5Bakf/AAAOJkqMsKABbxLdj6NNWjdf8z128jPX0qX3vX5kyhgXSPy/x93mBQAAAGxBmkZJqEFsmUwIT//98QAAAwLGbvramqtjIw0AIwK2Ptu0Ud9hBgSrGMxkDR7VdMtgVm8ZkM9IF37oHn6UHrvHIvAVXWpVUwzSYmtZpMyfcZ5JyH+6d294s5oMRpxchjzueqvRfjjpRY+dZ4AAAAA8QZ5kRRUsI/8AABfgh7cNuRyt3u4q6gxwCLFe+EAsuca3gnZZH8OUWarw/h2BMSQG6jYCfhyQqGH2RzZ9AAAAMwGeg3RH/wAAJav4W3gYVcgAWojZ62WRewZyL/9eWdt0Q5T2eoDmf0iHRC2N4dfGIu3s+QAAADMBnoVqR/8AACWyK2H4AW3V827u+DDpdRiRPaxCFR8pjaZoy66t1+V7spZQopgckiXJiXkAAACUQZqKSahBbJlMCE///fEAAAMCxNQ2q+A/EBnv4F3cePOKp+X4oIZxNYEsYhDZHQxTlpfyh46r1Ia+Qk0hIA1X5G0CWyFy79mSTFIdgdvTlMupSI7Y0e+QgMWB4w2wq/o3w3YXiGRp+QTOgzKlOJMZfhR+efykHr9ofLefKOao0iwQRZa4NK3A0p1E9ws0/1cNuqnIQQAAAEFBnqhFFSwj/wAAF+jKlGSN+KAL+4p9bPlZHSKKU/v0pXQU/98kMzTrqrtAZvh/4ntjWc9DMp49qsv+XGS31sd1gAAAADoBnsd0R/8AACW3/vQvcE3AAs4m99fEMI1qU2WwM0Z+EaHJwT7Y1SQZDopOuCzFDipeAYEN1jKGYXDtAAAAOgGeyWpH/wAAJbHMDVZozOabQfoRxYekUNsI9fNCsn97pI6j6IyrhrYAGz34d36k+CfoOyWav3+ns+EAAABlQZrOSahBbJlMCE///fEAAAMCw+z+rSTNZfBdoFAsfgQyavJYYA+moGTtuFxOpfKX+Adm3fNRAqXGmaD9PS0ttwuJYVHd6H/K/1xVw40m9dCtOgQdWr3VhvRiR7T1kkOHg7SDuiAAAAA6QZ7sRRUsI/8AABfgh7cNuRyupDs22dhAkALeaBgfHOfXiDsehFpisTJJads6O6XRzAAWvgneLg7O6wAAADgBnwt0R/8AACXAmTJ/uUHcTj/ouQUUxbZIpCpAAuf3b5EQ2gGfupIN5FiUJZQ48vbXkJnO/N158QAAAD8Bnw1qR/8AACW/BCcSy49yED/+SCAAlQqsyjxPBuwZyMSQlWi6LrvKSDCEs2IKgEaDE8Xh18a1FegBJp59zacAAABNQZsSSahBbJlMCEf//eEAAARTW9G2nXXlIFIMIEgATtrEBwZLCOLf9Imnc7+Z5so0ByBEtb4TTB76xp+9LXwg3gEilRK7MyxkdFMbNUEAAABKQZ8wRRUsI/8AABfgiIvfOJL5MmppsAAThtp91N6ENNt9DUG835LkDzCPbgpYa1pTf1hZdhE+6F1jYR/caYkWfMM8qXp0qoKS2z4AAAA7AZ9PdEf/AAAl1xUAHGkVCNYFDGd6iGPRW4HY2yCfPNsgOr7NR5LtVYEXoDY7JDjdrFhqzg1eiWz+Gi4AAAA9AZ9Rakf/AAAlsi2/BwAXDZYb1oIHkZUR8UzZ8E+x1Jir8s5dBChoufL3GwHLzJ4FJ1jJb4W04XnJNYpzpwAAAFZBm1ZJqEFsmUwIT//98QAAAwLC6O1dAG35X6EWVBHAx8evTI0NoqS/YFho0qokKmL/lbJfXkZ88NvANeM3lc+fbECIgPJzQ66IPN0+5XZu2tEX4ZEFoAAAAD9Bn3RFFSwj/wAAF+jK0eZCNE+V9toBKAE0wgPU/JPkAkQ2lY5ipYiMiZ1E3sbCNOIiP5sjNDrQ/G+/EBlaz4AAAAA6AZ+TdEf/AAAlrDerw7AUtj9Nac7/4ASbpO0kHPaQcxr9uvW4M+deaFis77j+Y0zD1Plq+SUurmcQIQAAAD8Bn5VqR/8AACW9g/jPyGPfwASxTb+m5XcZz5+PQ3Zm7AfkAxGIXIB/q8w/iTDYylGLzMhLaTEBX7NYcH3SjPgAAABYQZuaSahBbJlMCEf//eEAAARzW7lutwq0DAgBlMelmNcJPv2vEWVagoIyptb4SlGABnbU+70FFnC3JO1zzoQKm5vbre4u278xmTwoCLCtTN3NqZESxSXQ4wAAADlBn7hFFSwj/wAAGImDFqjY07QSktONWasc2eshUqxrgCRHEIhDSC8GUIOkq9STLVEoOhDLfleXXcEAAAAoAZ/XdEf/AAAlt8wgCH11zeB8QvNf3uR0YaR+x1dBxZ3/3UOa1tPpgAAAADIBn9lqR/8AACa+9T4apA+XYP1okjZ0yhxQM5ABIoR8xAvtm9ICx8kQL85my1tV40iruQAAAHlBm95JqEFsmUwIT//98QAAAwLU6NiACuHrhrztkJtMRP64xLtHDWikmZ24JochxNdz2+JayembhmhLR6ja26M3cE4mi9dyRwLGnFoGFD45eTB+3uy1Y3c/ww9fkLFi6Y9yDKy0UhG7uJpvaDO/KrP1dE6i9RYFChzQAAAASkGf/EUVLCP/AAAYf+weonSkb2+AALrUZJMs55jynJaHL6r6raHSVaVc3zpAE86SyMg4NTYz8hluQwB1ggNUhhZHgc86rS7jrX2pAAAAOwGeG3RH/wAAJqv4V+ASWbeiH5RRjA687giBWDdvAC59W4rSD3aAM1MgBeS+JVLt2blpP9170suCElRZAAAALAGeHWpH/wAAJq5eM4IvtBfq9B6XSAzqYv0KZvYQJTEQxqJz9Hc4zp/HJ3CAAAAAY0GaAkmoQWyZTAhP//3xAAADAtbR57YAzfLfrAs0rV9TMD2BbXCzUVmQWmH3HtLh3rO8J88FXmPOUP1UALTM5IG8PpDXD0VBb0YXzVRQJ6Q300JXoyazw3dcSmY2RNSgps+NRgAAAD9BniBFFSwj/wAAGIjKlGRy4RblDXVgseC5gAmagUgKkjiwn2xJwIkiEUEysV3vbdZSGXrec+XmgODS+fwXcIEAAAA+AZ5fdEf/AAAmwxdShDovRZwi5u4EXYjooIAEp3DR9NpaI79DWMoU7C15hqixD57UJVRF7vYRIZfD8GQF2fAAAAAqAZ5Bakf/AAAmvYOBygD7ECd3PybyPl2WDAVppkcEMrSKIq3MXFTyXkSdAAAAfkGaRkmoQWyZTAhH//3hAAAEcz+jn0HeLkIohHu4M+z6mNHjLynOGkcXFwo0RiUNOzhA1FG7aUgMB/Zs3OXgzhvN2pSpLYrkN0Yngc6iJYKB7lg3JrV1LB2UZ+5zIazpDbtEOAGjnCt2lGk4apABmd33qr7JXXJjAsAlhZXk6AAAAEBBnmRFFSwj/wAAGIibYuSUG0dwAfh4FH+2PRLyfbEt1Rh0rhFUKXT4Rb8LaaLXgtF61kqs7eOfBYe1S+5f8fuFAAAANwGeg3RH/wAAJsCZ3ef5T0XDoAbmO8huFZkgq60BsRCuYFk124cAIwANCu+cYTgNrGDTB/7+Dn0AAAApAZ6Fakf/AAAmkrupJX228D1JKXF0S8OtHCpY6T6sjEtB2gWB9vCwruEAAAB2QZqKSahBbJlMCEf//eEAAARzSGYoA2qRa3e5fH7a7AzktHqHakhOttIEQf9e6FMUgJvwUWVHBRbNmjhupaVhEGH5njZ9bxte8HKcUpu/nGcFf/m46NVcwn+blmIcUWFpMfzWZsiPhabmPLuBEXS4ayyPZ1dfawAAADlBnqhFFSwj/wAAGIibfpSZv8JQOKe2DvZnTEI/noKavAXXCrO6T1adJTvzfNBsAGQ8WQqBBN6FOfAAAAA2AZ7HdEf/AAAmwJrMaZJqJdDqEeQX4X9EIF0DBVglSPIT01CkICZc4ABD2iCcJhq5apGtErhAAAAALAGeyWpH/wAAJr2EHhJu5witSSurDMOkSYuOEWOJnPxFDYNWmN5/Kqh+R17hAAAAckGazkmoQWyZTAhH//3hAAAEk0gy3+oCcKAUO9n7x+wrD4wEDktRPzE4Yd5EsT1ET7lTRe4L3yOWkgUNqozRyPk8boRDEizC4f0jIHqBX9qd1c0xmGbgyFYcx/EaNG4X1pUwjYiq4zcffnVb+gPeSqxogAAAAFlBnuxFFSwj/wAAGSl8w2Cu7fZnlRIzndMl3OqaEcUAoMU1hHhOzyglIikre5XKQky3fWx9EEEVRWM1K+ttXj2EFQnZTQKTHOhkmopMnLwc/RC1snZbJYTXgAAAADEBnwt0R/8AACbAmswO0JhlZB+PLYGZL0OWI5dyLxIoJ1A374s1MFOksK77/6lt0j3BAAAANQGfDWpH/wAAJ8VTyzzNvcs+sCAWwaqHWeHPPgZB0DElnmYCFU4Q8LrmwC4J/ffuj8zLlx7hAAAAa0GbEkmoQWyZTAhH//3hAAAEkz+agTVAUEWzxcy0QnbRJ5+m/B2YyqDCgDi72j486YgMxVL26strrzfJ5pJLzc1XaWvxlxI+RKfJq2OJmqRyAcwgVctTrVPpSAAWdNnh3BFNJif1GD5UvWGVAAAAS0GfMEUVLCP/AAAZH+votDzTzF6kFxfkxZ+uYsaOYy3QSRayhVWnpwT4rd37zn0iTP4sMAD90iWNRU+Rl/tIzmi0wxXUkjekadN7uAAAADMBn090R/8AACfCtCKtgBwrcQub6kQBCTt6IZv+/HaMJNOqfuFxH1zjYK8NC/gNx3W/3cAAAAAuAZ9Rakf/AAAnxVPTPHqIQKlxY4wdYM2MvHrsj4MQAlz5E1iFtDScmJR3xcbFgQAAAEVBm1ZJqEFsmUwIR//94QAABJgiGzwNiw65CkVXSWr2qrXZKy+9uBtYlCskpb+G/WkMMMi2TYZuWENlBdL3kwpcu/F20oAAAABFQZ90RRUsI/8AABkom26m5g7wbSnmZ6bIrc9dXfZNJtY1nrj4WW0RGScWFsbNPgAnY9rd/OwoyTzVD4WFMjom2noOMR1gAAAAJwGfk3RH/wAAJ9tXJihWylRKTQ0zd6hejzTXSku7SGbybOklPej7VQAAAD0Bn5VqR/8AACfXqH1plYJks9dfoL3TI3bo99cdDZEYMQu7irkBUiADZ77Ad1sWYtaIbB0Fmhayvc7r1DtUAAAAdUGbmkmoQWyZTAhP//3xAAADAue5B89AFw4PGCXcBRQkKF1N8b30PQd3jIFdnRPUrw+pHe+qoYqpP9VaRqEHdP07b9iVC9nEyyjxEengBh6gCUNvDPqOjOdEkJfcgTD/4P6WivD5GWeB9gjRvitJFzBqH3XyEQAAAEtBn7hFFSwj/wAAGR/siZOK8XoCKM5sT5t6M/zY2mwzqzTzMrGaPB0bAW4P30HI9SPAB78ZLxlt737sZZzwKYlq+16/0kzngbWvGxcAAAA4AZ/XdEf/AAAn21f7XwwJK7V00YHLIOzAnFoGuBf937U51zSM5bJKKAAJaGnwHA2qfiAa4oVN+LAAAAAwAZ/Zakf/AAAnxVXiIfYx+p5DqdThuWCSSgkyp/0Fs17ed1JeGuDdCvzNX4+J47FhAAAAgkGb3kmoQWyZTAhP//3xAAADAvpO8fXgs/wCSMTCWLnolbtZ7bdLzVV2NBfFvV2H58Wbx0Ut4Vg21/tzQ9yVZdWj09E6yZWBER9cp54yyPmZQMsYP+6W62x8t++gROwyQ9wspCpHKtxrYDCo8zKenR+LhHs9o87uWyVCkLDF7HfciEgAAABBQZ/8RRUsI/8AABm/7FOAuHyl4FKwAGkB6Jd0gtwSLxaQ/8f7VbYOILoFuscQ7VxJRIV1ATKH6y7SOBMqWxjw24EAAAA4AZ4bdEf/AAAo/ioEuHMABwyF9ipjinyHFLchCmlpuiLUuuZi8+SB2o+qqVwTWtlb0kxKFHRaxYEAAAAvAZ4dakf/AAAo5VSDUxSAcvk70GDyCZPllwBA+0MlgKrsO8ofWz1kA+OFf8Wi1sAAAABYQZoCSahBbJlMCEf//eEAAAS0txchu8fSrJxwE1vu0LFwYbzYsNKtUAXolEv6yOzUjCtP4bjUbkfQCYWG1U7KA1+uYIT3VWD9VQoMBwhXMRi0vBlh75jiaAAAAEdBniBFFSwj/wAAGb9DvQtbYVdvKFFC0cyGlkxBreHTO0D9lOvRMjgaCR1TvNIRiugklu4jPEra6JdE1NeEVh5WO53bK+lPcQAAAEABnl90R/8AACj6nRx8N5fDWMdwm3a6L+dQRjBHgSlNnjHOc9In2LnTaZVOOVABOqTqU0TcNIJCwG7ej0uWMsCAAAAAMQGeQWpH/wAAKOQZl8kqDtE0wmngYaFSwVjf9ZkXQYIdClt/mHNoHxpFQtPWpTaz0wMAAABvQZpGSahBbJlMCEf//eEAAAS2QAwhqrwCaGgpHxcEvxxTmDQdvGBLtKgFCLAHMeukUdKkogFbB+iEgBBs1BljqQIT/pNx7oq3aXX+GhHUNgOG2XusCtdSivfN1xDvxWcE8FpqBWi2DaewLmjJSp+AAAAAUUGeZEUVLCP/AAAZyF9ZAAHGCnnKjj9VMzohKhSt9x86UacfFODua+8pEwPEUp9gaMOQjIs9MXEJYpTQ9eDbQ2MwkLGF9qAkqqdIjOSGgZWGBQAAADEBnoN0R/8AACj7Vr7cRhZ7txfj1t5+uPPK2y/DtFfDH499X6mOOLjvuABbMqEh9ZTLAAAANwGehWpH/wAAKQMVxPAIRDxWbRXHT4nH0b/l+H9ifoyH0VI/VzJbhGtpn462lsLeEYKI/+ZGqy0AAABcQZqISahBbJlMFEwj//3hAAAE01u6nJY5aMyaMs0X2FsMsBKAVtPj6jpQYxVFezxtmLE9IYXNK8pNCnRN5DM6CW1/2o6ovMu80kYNvTt7DdM0x4bbHA1ki9Al51cAAABMAZ6nakf/AAAqGW44rBQKv2/J+HIqTU0/20ddd990Nk+lACvK1W+fm+9+chnpHy1wUU+TH0FkIo7WKEkg4S/6KAEX0X8TjQvTGgOGUAAAAHNBmqxJ4QpSZTAhH/3hAAAE1zOhyy4BCPKeme4QTvaLwjSOvyf30vgjICTCPEhD/DdZ34ZqArjJHqoC1fKZorMvOieIa/4maCtTmfPe4QEn/7Tmv7N7hGEW25FRsNrG8LtSfUzcvByXI834eK/dahlROMyiAAAAX0GeykU0TCP/AAAaaGMr/yeIsgB/eSDVvLRYos9vBerUwPF3vCEmjU1fnGwD/A6gSnxD2ecgWYezf65ZWihr8vFbvUvy82784oO0gjSNpEkPbsyKIcwiadIUqIa8dabLAAAAQAGe6XRH/wAAKhqdjWIn40XhPeGNZvnhyLIMlk5PoNVXR0XeAA9B0impDNG+dg8ALuThKTHP4AT1+cEFCSu71gQAAABEAZ7rakf/AAAqFzNFVIR0E31IeLaRV59ikXlwyYVJ2Mvtiaj3u7He0glY7IAM9OGDeazIjQVoXN7q3sNxo0RTIKdpGSAAAAB2QZrvSahBaJlMCEf//eEAAATUtxpigCgDDal88FE3FvPs0L4JyX29sPr1BPOLT6oGn5b0w4ByC/OM1a3KOjVaXFuRC2OwJrezVY2dOBJOkW52/5qhQj/6WDxXvtsQKOiiCyC9ryrRupo2I3mIjuLSO2eYkjZLEQAAAERBnw1FESwj/wAAGm15hhiRySj7BYaDKT3GEz9bkWteaycOJc5GR1vaEBDHORf/6qgp0UIC79eFrAAIV7/kDICe6jNTrQAAADQBny5qR/8AACoXMyc/F6v1a+uHczC9SZ8MbgayUqjyPDFaads0EYlycms9BVv/UYWD88DJAAAAXEGbM0moQWyZTAhH//3hAAAE9ZA1ewD2IAUhml9q7X0ujCZrCHuxIWE3IRd2qQaADIFM6F/U3nBSHIhan3pyw8i7fPqnzpQ1b4SvVo8tXkzV3osyAY/WxS3dJiGUAAAAX0GfUUUVLCP/AAAbCYbOHS+xTqFlACWNpZOPHt3kvQBwx1LnUlHcqNPHcY4Arlvb+2D8ea68Oz8pk8EokbIewlspkr5QvUxWKBySpoyyb7aphwNYsymS3ThY6xVICt6AAAAAQwGfcHRH/wAAKf+fLPr6Tigw+3GJ3ufQsmJHusr6kEGfzCO92n2iXjLZggkAANhh5PC8UR5nSMHPk4HR7ruOpZDehkkAAABFAZ9yakf/AAArJVR8DgQ7NMeZjOex299rzWuDNKj1eWo2dC6nCQNAOUfqojNQFFAQAE4vVVZ+T/v7AVtrQeCPHZbn16GAAAAAS0GbdkmoQWyZTAhH//3hAAAE9Y4QDt9JQ1MT9z9m41aa0YsWEL3VyJo6CyNj1Ai7eYzz1QBwZZwaHIvfU7FyHeR/9JfIMkNJ+KIviAAAAFdBn5RFFSwj/wAAGv91SYg0SMp5wwz7uRQkBh9vF78Qir0cOgBGU7qAD+1lk2y6QNXjIQHkLqhz0aG0FcwM7Zryv4l+0qYUhxcMbZtsnwHNkVn1lipH1sEAAABAAZ+1akf/AAArJBkycPp8yK8vKALAiveyZzK2kKaJg8UQ2UdTPsD++oEtIwA0eJOLFTJiToLARqcjD6vytShXCAAAAGRBm7pJqEFsmUwIR//94QAABRmSXn902hUA/dckmswsAGQG2/gryskcowoDoNSxJDnAs02DAkU9t7J7roe5TfDCu/s53EN3JrP/JceHj2Yeb6EwpW7ufs51pa3BSrMYnwIgUx3FAAAARkGf2EUVLCP/AAAbqXxxSYLnP+tmI8CKga5wXTaa/aACTbLvHC6VfuUSlOJCJvezZVGdlnSgAhTvF3uptWvRXojSHNBzJ/kAAAA5AZ/3dEf/AAArH56gj9fBp4hR4jLvp/kwRhGn4TezMD7slQ4Wg1jCBsptvtvHYsAHxpPMkQmqhxx4AAAARwGf+WpH/wAALFls+5Y87cSyRcdtwAbbnjtXgEbQ/dmz5TTjMwAhzk2g6Qt5nN1u6SW90jlZPi2i53qpxzOhopbYB/QYiMY9AAAAVUGb/kmoQWyZTAhH//3hAAAFG6ch5gFtH6JqaaUZeHykp2dySJM1o8LL4FKbCIh4ibADklO7Dg6tL+h6Yp9sn3VHwA2G/KJJlxHikiw1XNyEC52hgqAAAABIQZ4cRRUsI/8AABufdUkKq8pFWzVp+dAnlKbC81+O9KwlasBb0DB8r+rcddqRP1yVH+8s2D4AEOmg5K3r7ieCstuJS79JFt59AAAAQQGeO3RH/wAALD+e2lUjamQq3PYsLDgSoK7FoKXE3D4m2zYawv4Wl97zmHfirUuvxuiwyGhzoZWSyGu/w/upx/nxAAAAPQGePWpH/wAALFczRVRzBeUvUm3uWYgsUx6n5hct/pkS0wbskdskIoaTsR8knB+z9mobnkWFt2xqYnuP4vwAAABgQZoiSahBbJlMCEf//eEAAAU9k0mjTLLRUBANzYk1kaMqC/SrkVIN3W9Do62PPtnr2iXWJ6RkE1iKlsAmjSJ6tfA5H0KuoFotZMi/csCYDi0RMnuoegt4K0a2Nytc8wVAAAAATUGeQEUVLCP/AAAcWHICgjgA4sEOHeVTT/y5hr40UzZMUkFZo1MHGKEQJ0BZvvVEcBD3iqbadxYYbHG2zqjirYywdFkgijYh0aqvJoGHAAAARgGef3RH/wAALCeP9mJIvK65dgY7izT8fV2m78hG435lVYCLjWDZ9qbRHbPRIN66ACWqd9hFdcd/e7VMVM/4hza0lNQxMW4AAAAzAZ5hakf/AAAteWz6W8NhdO86xEuJ1kJ/MCT4KIY/sM+uma0lMkMnl0incED3x9SZUqvNAAAAc0GaZEmoQWyZTBRMI//94QAABUGls8utg0CWem8Q59XYjft94vJ95N6IZS1yZow2s5ZWVQs3Jm64CPSpjADvksSIlh8u3IEVgxGJYLfrOQFWEchqnqIPaUM9kpJrabBNm1wy033TwHXxO70mBgdU4zDqm8AAAAA3AZ6Dakf/AAAtgyVrwoQShJk91m8LIi7v7XjowQYWtTjO3Jxhed5vDTQ3tJ66Q28K+cDcNzbWpQAAAJZBmohJ4QpSZTAhH/3hAAAFYZPTqLPWq/JgrSVgAuQVcfqDi6gr4y2zf3ZfaO+ojtrv1B+VV1Z7BogiDYcD8f11cHU+/F0NMbEz5fK7FcrsyxYEpzLY38MFflz87TLoiL4+m1hdZNM4mlNlrMA6ZAcQ1wTxdzVZ5dqrvGhC4EModQXbzS3zKRzNs9Ih2wljHMJxtPs7zYEAAABXQZ6mRTRMI/8AAB0IcgiyYLZsLQf9gcXIQW9Mf/kqAARk3rRn8cSNsyUxvUUbIxxOeZ3DsfZt6q61RFYdtAxddm3fK+fWFf/2qwTa4IVZ5CEOjniRI8V5AAAAQAGexXRH/wAALXqc7UjkNcvWjy3Me4Yamrt4Hnya6agbPHwrQC2+le3fjKQIcTbMMZzYsAJX0L7oRCgCrRdBq2EAAABEAZ7Hakf/AAAumYGRSpjwF/43cPpkzU1ub8IK5G0GvDZ+3mXSzuDxaEP/kLbTh1k3bOhAAiDoM9qjBLacVj5ScEZ9VsAAAAB8QZrMSahBaJlMCEf//eEAAAVhk9Q6GBlsnsZXS/BACzaIcXT67qPIqFg9vunA7mRtVHYLBZXmds5WrO9Ac/GL8HNDsb4Aurg1P0Z+Zh6WKYOG2iWOu5+jfqePqfioqVGhZ4mVNJQMy7Vwq6T4TQMrjbzsutU7tnnkw4yGgAAAAFRBnupFESwj/wAAHPyQZILYapECgq+E5u1VoABx28xzQ3bZ7elZYLTes5jrutSVWJaBYNrkbb00/2K9WOWz7VEm4QQEyukmkMyQpR9HeQ/0uBzwDHkAAABYAZ8JdEf/AAAump2NYcKVnbhJHiaIf0lZkRgsHaE3H1fMdVcweHcTPKVRY/KFaxEnE0E9TQQAfb3wA5OdkPA1M2OxHzpW4Dm3753Uir7eeNyMiqmjHgUv4AAAAEcBnwtqR/8AAC6CnJ/0ffwJ4Dn17fZ884cWBXtqasra1SY6kRYTshl2B6zIm9U6LIgGP5ijMn93qXAB+5pN8SyinO18KMKQQAAAAIJBmxBJqEFsmUwIR//94QAABYWT0zZz8qgZL1S+nQ6XweRjptZDbFa/yG6WlRLuWf4Zz8HR4h29ypPg2oHNQuO9oRKaarmgp3/jQL6pEY5iQ9zjnqg+ybQ+GU92rACbm68KDBIwOAchf6kWCuu7B6J16d/M/yoc8hCcoBtFMZlsyp3hAAAAR0GfLkUVLCP/AAAduHreIU6kZQ70Otq64FJYXDK3Z6XaEYykrLG4nqfIZJx4j/oclGpkwOpyIDyGuJ0tV7/l+mOUOafIW5OBAAAARwGfTXRH/wAALpmS62DwRXM8AMGMu8uanalQUfvLKZJbnq7eRMDCSgd+xEciPFWMjX75RgALmF3imXa/OGPT3xYDPlUr4YrdAAAASAGfT2pH/wAAL7jdoRFfIt3zBkLYT/jN8pK6XpSvXWC+/UYPpWWsWCcMS6dtyF/CftdewNuT7JnT3IstTACwObzyMfgbWlcsOAAAAGFBm1RJqEFsmUwIR//94QAABYlTHOVxvgGpSCNU4pHJp5/8X5h/MN8fPBnM9+3OGs8N8C1TR22hCHHL/XC42rVHHPDTz5hVDwoT2bsYTN+kb3n285bIkCyUOeFAsl+DUV/BAAAAOUGfckUVLCP/AAAdtwhB0hamlczDntepcHEkWtkDsedHfW41HhxH3gUphKHUVdJi93JNXvRQpa4ipwAAAEgBn5F0R/8AAC/RfQ/efhNiobwQrBrVzgQjihVusni3gfbmjH8GlBBAA9rBbjtv6nZB74HrEyYZ+EAEZFLD5328Rw7qEyuZRm4AAABHAZ+Takf/AAAvrF38xynRZl6czQplHnQvrX/FODw4lEffs6VQrLaVnPShaOaKFMABANEErEVo4CUi9DABuUvscu5ir0OupIAAAABPQZuYSahBbJlMCEf//eEAAAWplOg59B3kH35W52Dx3zB4oDCMjY9IDjw8lrGZyfmuuxtQHhHiwPPrG/wBzDYmbwAfULIqJnwH6fihenGmgQAAAC9Bn7ZFFSwj/wAAHlyeZILYcKfkuoD95r5P5knwVTKXYsZsK4ILyGhaGJKAueFtJAAAAEIBn9V0R/8AADERfNekkWNOGvAS4aa0kvH0/d6W/bOFmhGRxHuxKwAAmRDt2E6hCpwTt4Rk79zzF/nh6aLfGjC9KL0AAABCAZ/Xakf/AAAw9xe29k5sbGm8NL7g2BaEXvawhzGlCaN5LyXa1/v+uAAgGiG20k1s4wrfBi7u73WWcyYcfvTyjYlxAAAASEGb2UmoQWyZTAj//IQAABYUTwP9tWsBXTxOSKUye1FJHE8dhUrjMusM9itYcNxl3IzbZ2amVQyc3kAAmUM3Bmk+QIBAK90Z4AAAAnVliIIAC//+9q78yytHC5UuHVl7s1Hy6JBHUl2zdg6GgAAAAwAAAwABqizOzxtjVPNkAAADAXMAQsJKHkHmImKsVAvvN4HO4ARdc10Yt4Gj9uQRgwIYN4AUHzYeZIETC/qkr+fpkUsQO0exSEbFJ/vYpRoAQYmVyV4Xdm2O2xfsjcMtnutR2yteN71zHbvXVyfoBtLHkPA6Wfc2DFtSYX3a9+qVxqo05LHTLND7Ty2hQGB01lnIAMz47+EcbVnwQbHRB2+oz48Pw/8j3I+KsPMO5UYzE+z26ARNywM/BBpf+2cWk9IaZyNQghwG1CU9dR1dsE98gnR1We85tSDHvejYHznNmvDShq6SWa1o3m/M6FEdKaxYbELrw3cIsuhSjODp4o/yuB94H4bx1x+vSCGI5GCXW2HisOULoJ80GxMrfoZeqTEV1MBJ/3OqQgAUJ/tAxWIqIXFRRGCyFGA/3+OCG/4AWawE6/4jzzzK7c9j5rogBKmRCZD+WZlcWxgFpf3Zkcz28WmOOMNE6a8I5M3ghYcTCA8lDCt7U3BVN7M8VI9Mi3fgf4AhsAEfxQvw0uvY63+0ctAvdlCabN1Kk4eXOYFPP6KythuRcg/FpW9/xDznMVXjhc6KWgHnebu00x4461w5nKh/UNWp4z/hQSz63tfaNYF7OW34WE1mmxUMChmQn21Ly1Q7cRl0iySz4SaFH8MGw8HjykfuojuV/CjQC2/9aPSQ6E0rodnI40jtAGYP+ibMLX1e6S0UJNVP8qXYKnmUrjbxanG/bj9z+9mYZKg+NVRObuaMIaZt5+6CnO9qWLABjdaWJLaAAAADAAADAAD0gQAAAHlBmiRsQj/94QAABc4i6pAAbRnPvBk2KyFTEMw/7okOM6nSPhZO5NiVXHx/orMXlxsSEgo8CPRI8d/on3RRrne8PxcnFgTDkvJlZ8rFg2ZbObbAtRmhLPwi7MAptvOR46w5rrO//6zsLZCiGmnIejHGTzLK7o+7kyKsAAAAPUGeQniEfwAAH8rhCqud47PK6JEKaOai/wCJWPynImzScJv2yAEVhKlYtfhYylOaLUlWnHUVHfEjGHwYHycAAABOAZ5hdEf/AAAyM47Gkwkf9Um3/IipEzJAYG+3qBNCRpYzTpjCHpbtxb8BiKPbABME98jAGfW27Kcv62PgoWbVL2DHJVlx9VNXa+GOV2y5AAAASQGeY2pH/wAAM5I7fU384vISRLp6Y9HEijtibHa0/fZXbgAsS5Twy6ujmE/ERQZm/Y4WzBpkAD5S1j4lu0iAbDyyL2bQqwyDzyAAAABvQZpoSahBaJlMCEf//eEAAAX0Vf5igCg2RakwZVBPzPgR5Pm7PmtNTEJapnSK8Ay83F/GzMPPO3XhFma9U1LlhAKHS+V3nfEFfsTjnet/jmZSY3kdWEqLPOcfy+0M6gaGZQB1hiD2ox1yytTRBuLAAAAAV0GehkURLCP/AAAfvKxyk1+O6rG5rJIXnQHeCCRX9jnTnAbTFQCKp8cKKYFS5vTtQzreUYUFCLXu8ES97vMY3wEBCC7LJk3Vfi4Eo5TBg4FkO31frVdKgQAAAD4BnqV0R/8AADOR4jh0vOWJmklhZ9c3UB/Bf2UMpsTRL/UU29ATWWr0qL04V4k/VjJQQEEluFG1QO+aLGcgfAAAAEcBnqdqR/8AADN3nzjob1ny+Hgz4oQTzhDKWpZ+WFKrNOjnruPkbLA2zHPVAAQPW01pPWCwh88d1Qw1edFhek9+B9w2dOdNjwAAAF1BmqpJqEFsmUwUTCP//eEAAAYcTg/pjrOPrHGG5zt7mRhalc+qWqdr0gq98q/rTV8eAIBGykhPNtmmifTi7fdUNMSbgCt94ubwBGeS8BOqLqCHhUsGb35l9IppZcAAAABOAZ7Jakf/AAA00l6WfZwfsCrKsbTXvCdHlybnuN9XY2u/F0RoB/lM/b8ZbBGW1LNJZn4wzoAICqGCL7CCikbUZyHMDZ+jalXS/KKc8dazAAAAgEGazknhClJlMCEf/eEAAAZEVkNigHygYDMDmgvDTHjp5OLkVZcU5nBWmnoC06LUe7CXSPGM861xR0GPkUfeXVQ/4s6sDP9hi9RunQyACouJp0rChhgs+HvzDT8I/7GoJjetWLj96PG+kZvr6BmmIPaaBAOnFpMoBJdWqy1sc3IrAAAAREGe7EU0TCP/AAAhvIW1WsJ76+cr8DL5bnINQoiwMMrj8gkxjOCa4w4Q6sC0nMfISferMAFxTOjKuq1xxcCzReKdMtQ5AAAASwGfC3RH/wAANNHiD+j6cUEbgvBpUdw6Lk1uHjDK9X8aBc7wbn+mN6yEAD2s50o1fMyhqiGd04e60w+Vn070LFnOSsuWwnBxR3kBLwAAAFMBnw1qR/8AADYSURD+J00M0YyctptpuRpkKwxLr7qZM2jITT+seRJpEkd58MxZMQLl8wuOjzOO6eHjHAAB+t7ytbuAQuhmDUSpYaImRnWqeLVi4AAAAHdBmxBJqEFomUwU8I/94QAABknwKF9Rn334sGAHJu7KW/9qfOZauVm3f7CVGvmFztBnwfvWWU5GpU5MTQMMqG1ugSsN5uAJVamBtw4bQBUopUfCr21uVu6yaYEq7Tvv8eJm3ygvveVnnFBti7hz+hyS1hYLWwpZgAAAAFsBny9qR/8AADX3nrb+fCZCUhEP9ChCq3iXSpHnlLqQkypYL2arO+Ylx+bykYBuWAKP30ACZct9H0X+CfcQNnTo2U0Fq8YyKDZUv0jxPHSvKrsNejFJyzZiGNMbAAAAgEGbM0nhClJlMCP//IQAABkQCDlAI+uZvhqxwS/1BrU4DFlgCg1o75AQp/x4WIhHiMt3aoAvxKCV4JiBo67bAaFQAv/5OPEAgbZutvIvI3TPMZRaQeqz3BQWoNsMx7/KmuRiCQiI3cUO3oi7SuZ6+pOB+7dIgtNTylRzilMaxiFgAAAASUGfUUU0TCP/AAAivG7zURgNfNp0RpyVlJYPta/bH0pD5KDT9awQPhC2+nNnT8p9Lrg3gALi6MpxD8snAKzY5v8waWDpc9MGDzAAAABRAZ9yakf/AAA3N17EgCgSg2unPkYyuVtlXBDMh9n+yO1vyQ+NuTPcuboxXUqRKJEXUauHpyGIXKRVpQaJfZMF1si5Gf1aynWFO0Ot8BKL+pBhAAAAc0GbdEmoQWiZTAhH//3hAAAGcIIXfOvjfuEaBfcuwwF7i91VEBES+R6Y5lWIXMVIXEPN5mg2Iv+vCVCHHHz4nLsOp3jozN2bR0kUe65/FmjQwDANqBRxZV3UIkgWhtjeA4HzVkJXN1sSNmmyNsLUTg4frXUAAABtQZuXSeEKUmUwIR/94QAABpRU7tSfJCDvxku0KFaJeizGKPEc8INAFhELWd9+ucyh4y2f8BnGm+bpUmiU7UIa4AhRV5Vk/uYGeaM5lHYhW62AYWXcsMw6NE3hYLtOX/rrbMGurGcEP2N08wKQWQAAADpBn7VFNEwj/wAAI74qGnHqFbes5S776P0NN+3Y/HXuIUszq+/hoihYAH7oH7LW9tn7b2KPmDP46+OAAAAAMwGf1mpH/wAAOLJgxfvyWXfczboQkYPgw97mb/f02ggX3nNjrPAZ1w4vMM5GbBODMk2oOAAAAHVBm9tJqEFomUwIV//+OEAAAa8+jw1LNb/IzwqmbHHR6Lb2Cbj8+gA+VGEQJ+lISLjCxN1bnbmBSXQnrvZJz3A4bzReFijD8QHAtAM1zv5MZm56MWTP2yUYrjAW/N4pZJZCrtf/sLqXtUHJS7NiZPNl2evG/40AAABKQZ/5RREsI/8AACO6ke00Y105KTuXUxYvVvfDSbUu0Uvz9ORfqMyhfQgAkfkba6uRHvSjakjtLnBFFCe0n6t6sCxBeHY7FkdzDOAAAAA5AZ4YdEf/AAA4r0Yxc2giO1tWkPvubYiMGGYm22o45sR+AD9TGx9EUKqlSqDH3FzF8quRXr0hLJUwAAAAOgGeGmpH/wAAOJCzJMO5Zdm/m8tx5FDVf0jzO/H3juQbOiD3dKCcWEjg5vtcN187R3UMZXWL1imGHdEAAAAzQZofSahBbJlMCP/8hAAAGd9FE5exUC2ER6lMZxWi6JgJSmYZnZFEcRnxhsUHHQXjgvqwAAAAQkGePUUVLCP/AAAjs0kKBynEAmcF8fmgn1aZ67qlIy3L3G+Nk5QwZLwD3WBsO84ue4CNGerYnMmstZ3vvrbBE6BU+QAAAD0Bnlx0R/8AADivaxpc+GK3tJNgbT4wxxyaLA7ArN2LXpU50AE4vXnkOdRiSQggx1KGNxy7TznHQeTLFLbpAAAAIwGeXmpH/wAAB+/+JhcFAFnvBQMts2SPn9MYda5ZzbqjzGw8AAAQK21vb3YAAABsbXZoZAAAAAAAAAAAAAAAAAAAA+gAABYIAAEAAAEAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAA9VdHJhawAAAFx0a2hkAAAAAwAAAAAAAAAAAAAAAQAAAAAAABYIAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAQAAAAAJYAAABkAAAAAAAJGVkdHMAAAAcZWxzdAAAAAAAAAABAAAWCAAAAgAAAQAAAAAOzW1kaWEAAAAgbWRoZAAAAAAAAAAAAAAAAAAAMgAAARoAVcQAAAAAAC1oZGxyAAAAAAAAAAB2aWRlAAAAAAAAAAAAAAAAVmlkZW9IYW5kbGVyAAAADnhtaW5mAAAAFHZtaGQAAAABAAAAAAAAAAAAAAAkZGluZgAAABxkcmVmAAAAAAAAAAEAAAAMdXJsIAAAAAEAAA44c3RibAAAAJxzdHNkAAAAAAAAAAEAAACMYXZjMQAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAJYAZAASAAAAEgAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABj//wAAADZhdmNDAWQAH//hABlnZAAfrNlAmDPl4QAAAwABAAADAGQPGDGWAQAGaOvjyyLA/fj4AAAAABhzdHRzAAAAAAAAAAEAAAEaAAABAAAAABhzdHNzAAAAAAAAAAIAAAABAAAA+wAACLhjdHRzAAAAAAAAARUAAAABAAACAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAIAAAIAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAAAgAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAABxzdHNjAAAAAAAAAAEAAAABAAABGgAAAAEAAAR8c3RzegAAAAAAAAAAAAABGgAABMgAAACLAAAASwAAACgAAAA+AAAAmgAAACYAAABpAAAAMwAAAC8AAAAfAAAAQAAAACoAAAArAAAAIwAAADcAAAAoAAAAJAAAABgAAAA3AAAANAAAABgAAAAWAAAAKAAAABcAAAAVAAAAFQAAAGkAAAA1AAAANQAAABsAAAAjAAAAGwAAABcAAAAXAAAAPAAAABwAAAAWAAAAHAAAAD0AAABOAAAAMgAAAB4AAABJAAAAIwAAAC0AAAAaAAAAdgAAACUAAAAvAAAAJAAAAEIAAAApAAAAJAAAAC4AAAA1AAAAOAAAABsAAAAaAAAAcQAAADIAAAAkAAAAHQAAAEIAAAAbAAAAFgAAABwAAABIAAAALwAAACEAAAAdAAAAcgAAACgAAAAbAAAAMQAAACkAAAAgAAAAHQAAAB0AAAA+AAAANAAAACcAAAAhAAAAOAAAAEoAAAAfAAAALwAAAF0AAAAjAAAAIQAAABcAAAA+AAAAKQAAADAAAAAeAAAAcQAAADkAAAA7AAAAHQAAAFQAAAAoAAAAOgAAACkAAABQAAAAIQAAACIAAAAoAAAARgAAACUAAAAcAAAAHQAAAE8AAAAzAAAALQAAAC4AAABKAAAAOwAAACoAAAAtAAAAUQAAAEAAAAAiAAAAKAAAAFwAAAA3AAAAMQAAACkAAABdAAAAQgAAAC4AAAAyAAAAcAAAAEAAAAA3AAAANwAAAJgAAABFAAAAPgAAAD4AAABpAAAAPgAAADwAAABDAAAAUQAAAE4AAAA/AAAAQQAAAFoAAABDAAAAPgAAAEMAAABcAAAAPQAAACwAAAA2AAAAfQAAAE4AAAA/AAAAMAAAAGcAAABDAAAAQgAAAC4AAACCAAAARAAAADsAAAAtAAAAegAAAD0AAAA6AAAAMAAAAHYAAABdAAAANQAAADkAAABvAAAATwAAADcAAAAyAAAASQAAAEkAAAArAAAAQQAAAHkAAABPAAAAPAAAADQAAACGAAAARQAAADwAAAAzAAAAXAAAAEsAAABEAAAANQAAAHMAAABVAAAANQAAADsAAABgAAAAUAAAAHcAAABjAAAARAAAAEgAAAB6AAAASAAAADgAAABgAAAAYwAAAEcAAABJAAAATwAAAFsAAABEAAAAaAAAAEoAAAA9AAAASwAAAFkAAABMAAAARQAAAEEAAABkAAAAUQAAAEoAAAA3AAAAdwAAADsAAACaAAAAWwAAAEQAAABIAAAAgAAAAFgAAABcAAAASwAAAIYAAABLAAAASwAAAEwAAABlAAAAPQAAAEwAAABLAAAAUwAAADMAAABGAAAARgAAAEwAAAJ5AAAAfQAAAEEAAABSAAAATQAAAHMAAABbAAAAQgAAAEsAAABhAAAAUgAAAIQAAABIAAAATwAAAFcAAAB7AAAAXwAAAIQAAABNAAAAVQAAAHcAAABxAAAAPgAAADcAAAB5AAAATgAAAD0AAAA+AAAANwAAAEYAAABBAAAAJwAAABRzdGNvAAAAAAAAAAEAAAAwAAAAYnVkdGEAAABabWV0YQAAAAAAAAAhaGRscgAAAAAAAAAAbWRpcmFwcGwAAAAAAAAAAAAAAAAtaWxzdAAAACWpdG9vAAAAHWRhdGEAAAABAAAAAExhdmY1OC41MS4xMDE=\" type=\"video/mp4\" />\n",
       "                 </video>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Animate learned policy\n",
    "save_dir='./videos/pytorch/6_6/'\n",
    "env = make_env(ENV_NAME)\n",
    "generate_animation(env, agent, save_dir=save_dir)\n",
    "[filepath] = glob.glob(os.path.join(save_dir, '*.mp4'))\n",
    "display_animation(filepath)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Noisy Nets\n",
    "\n",
    "The main idea is to replace ε-greedy policy used for exploration with a deterministic greedy policy. However, to explore random perturbations are added to linear layers as parameters which are learnt. The details are provided in the paper at https://arxiv.org/pdf/1706.10295.pdf\n",
    "\n",
    "The usual linear layers are affine transformations as given below\n",
    "\n",
    "$$ y = wx + b $$\n",
    "\n",
    "In Noisy Linear versions, we introduce random perturbations in the weights as given below:\n",
    "\n",
    "$$ y = (\\mu^w + \\sigma^w \\odot \\epsilon^w)x + (\\mu^b + \\sigma^b \\odot \\epsilon^b)  $$\n",
    "\n",
    "In above equation $\\mu^w$, $\\sigma^w$, $\\mu^b$, $\\sigma^b$ are the weights of the network that are learnt. $\\epsilon^w$ and $\\epsilon^b$ are random noises that introduce randomness leading to exploration.\n",
    "\n",
    "\n",
    "Code cells below show how this approach could be applied to any of the DQN variants. We create our own Noisy layer on the lines of Linear layer provided by torch. For this walk-through, we will use noisy version on DQN.  We also train using regular DQN approach. \n",
    "\n",
    "However, you could also try noisy version of DuelingDQN. Further, you could also try DDQN variant of learning as well. In other words, with what we have learnt so far, we could try following combinations:\n",
    "\n",
    "* DQN\n",
    "* DDQN (impacts how we learn)\n",
    "* Dueling DQN (impacts the architecture of training)\n",
    "* Dueling DDQN\n",
    "* replace $\\epsilon$-exploration with NoisyNets in any of the above approaches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NoisyLinear(nn.Module):\n",
    "    def __init__(self, in_features, out_features, sigma_0 = 0.4):\n",
    "        super(NoisyLinear, self).__init__()\n",
    "        self.in_features  = in_features\n",
    "        self.out_features = out_features\n",
    "        self.sigma_0= sigma_0\n",
    "        \n",
    "        self.mu_w = nn.Parameter(torch.FloatTensor(out_features, in_features))\n",
    "        self.sigma_w = nn.Parameter(torch.FloatTensor(out_features, in_features))\n",
    "        self.mu_b = nn.Parameter(torch.FloatTensor(out_features))\n",
    "        self.sigma_b = nn.Parameter(torch.FloatTensor(out_features))\n",
    "\n",
    "        self.register_buffer('epsilon_w', torch.FloatTensor(out_features, in_features))\n",
    "        self.register_buffer('epsilon_b', torch.FloatTensor(out_features))\n",
    "        \n",
    "        self.reset_noise()\n",
    "        self.reset_params()\n",
    "\n",
    "\n",
    "    def forward(self, x):\n",
    "        if self.training:\n",
    "            w = self.mu_w + self.sigma_w * self.epsilon_w\n",
    "            b = self.mu_b + self.sigma_b * self.epsilon_b\n",
    "        else:\n",
    "            w = self.mu_w\n",
    "            b = self.mu_b\n",
    "        return F.linear(x, w, b)        \n",
    "\n",
    "\n",
    "    def reset_params(self):\n",
    "        k = 1/self.in_features\n",
    "        k_sqrt = math.sqrt(k)\n",
    "        self.mu_w.data.uniform_(-k_sqrt, k_sqrt)\n",
    "        self.sigma_w.data.fill_(k_sqrt*self.sigma_0)\n",
    "        self.mu_b.data.uniform_(-k_sqrt, k_sqrt)\n",
    "        self.sigma_b.data.fill_(k_sqrt*self.sigma_0)\n",
    "\n",
    "    def reset_noise(self):\n",
    "        eps_in = self._noise(self.in_features)\n",
    "        eps_out = self._noise(self.out_features)\n",
    "        self.epsilon_w.copy_(eps_out.ger(eps_in)) \n",
    "        self.epsilon_b.copy_(self._noise(self.out_features))\n",
    "\n",
    "    def _noise(self, size):\n",
    "        x = torch.randn(size)\n",
    "        x = torch.sign(x)*torch.sqrt(torch.abs(x))\n",
    "        return x     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NoisyDQN(nn.Module):\n",
    "    def __init__(self, state_shape, n_actions):\n",
    "        super(NoisyDQN, self).__init__()\n",
    "        self.n_actions = n_actions\n",
    "        self.state_shape = state_shape\n",
    "        state_dim = state_shape[0]\n",
    "        # a simple NN with state_dim as input vector (inout is state s)\n",
    "        # and self.n_actions as output vector of logits of q(s, a)\n",
    "        self.fc1 = NoisyLinear(state_dim, 64)\n",
    "        self.fc2 = NoisyLinear(64, 128)\n",
    "        self.fc3 = NoisyLinear(128, 32)\n",
    "        self.q = NoisyLinear(32, n_actions)\n",
    "        \n",
    "    def forward(self, state_t):\n",
    "        # pass the state at time t through the newrok to get Q(s,a)\n",
    "        x = F.relu(self.fc1(state_t))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = F.relu(self.fc3(x))\n",
    "        qvalues = self.q(x)\n",
    "        return qvalues\n",
    "\n",
    "    def get_qvalues(self, states):\n",
    "        # input is an array of states in numpy and outout is Qvals as numpy array\n",
    "        states = torch.tensor(states, device=device, dtype=torch.float32)\n",
    "        qvalues = self.forward(states)\n",
    "        return qvalues.data.cpu().numpy()\n",
    "\n",
    "    def sample_actions(self, qvalues):\n",
    "        # sample actions from a batch of q_values using greedy policy\n",
    "        batch_size, n_actions = qvalues.shape\n",
    "        best_actions = qvalues.argmax(axis=-1)\n",
    "        return best_actions\n",
    "    \n",
    "    def reset_noise(self):\n",
    "        self.fc1.reset_noise()\n",
    "        self.fc2.reset_noise()\n",
    "        self.fc3.reset_noise()\n",
    "        self.q.reset_noise()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#setup some parameters for training\n",
    "timesteps_per_epoch = 1\n",
    "batch_size = 32\n",
    "total_steps = 3 * 10**4\n",
    "\n",
    "# set exploration epsilon \n",
    "# start_epsilon = 1\n",
    "# end_epsilon = 0.05\n",
    "# eps_decay_final_step = 2 * 10**4\n",
    "\n",
    "# setup spme frequency for loggind and updating target network\n",
    "loss_freq = 50\n",
    "refresh_target_network_freq = 100\n",
    "eval_freq = 1000\n",
    "\n",
    "# to clip the gradients\n",
    "max_grad_norm = 5000\n",
    "\n",
    "#set up random numbers\n",
    "seed=0\n",
    "random.seed(seed)\n",
    "np.random.seed(seed)\n",
    "torch.manual_seed(seed) \n",
    "\n",
    "#init environment\n",
    "ENV_NAME = 'CartPole-v1'\n",
    "env = make_env(ENV_NAME, seed)\n",
    "state_dim = env.observation_space.shape\n",
    "n_actions = env.action_space.n\n",
    "\n",
    "#init agent, target network and Optimizer\n",
    "agent = NoisyDQN(state_dim, n_actions).to(device)\n",
    "target_network = NoisyDQN(state_dim, n_actions).to(device)\n",
    "target_network.load_state_dict(agent.state_dict())\n",
    "optimizer = torch.optim.Adam(agent.parameters(), lr=1e-4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_agent_noisy(env, agent, target_network, optimizer, td_loss_fn):\n",
    "    \n",
    "    state = env.reset()\n",
    "    # let us fill experience replay with some samples using full random policy\n",
    "    exp_replay = ReplayBuffer(10**4)\n",
    "    for i in range(100):\n",
    "        play_and_record(state, agent, env, exp_replay, n_steps=10**2)\n",
    "        if len(exp_replay) == 10**4:\n",
    "            break\n",
    "    print('Finished filling buffer with: {} samples'.format(len(exp_replay)))\n",
    "\n",
    "    mean_rw_history = []\n",
    "    td_loss_history = []\n",
    "    state = env.reset()\n",
    "    for step in trange(total_steps + 1):\n",
    "\n",
    "\n",
    "        # take timesteps_per_epoch and update experience replay buffer\n",
    "        with torch.no_grad():\n",
    "            _, state = play_and_record(state, agent, env, exp_replay, timesteps_per_epoch)\n",
    "\n",
    "        # train by sampling batch_size of data from experience replay\n",
    "        states, actions, rewards, next_states, done_flags = exp_replay.sample(batch_size)\n",
    "\n",
    "\n",
    "        # loss = <compute TD loss>\n",
    "        optimizer.zero_grad()\n",
    "        agent.reset_noise()\n",
    "        target_network.reset_noise()\n",
    "        loss = td_loss_fn(agent, target_network, \n",
    "                          states, actions, rewards, next_states, done_flags,                  \n",
    "                          gamma=0.99,\n",
    "                          device=device)\n",
    "\n",
    "        loss.backward()\n",
    "        grad_norm = nn.utils.clip_grad_norm_(agent.parameters(), max_grad_norm)\n",
    "        optimizer.step()\n",
    "\n",
    "        if step % loss_freq == 0:\n",
    "            td_loss_history.append(loss.data.cpu().item())\n",
    "\n",
    "        if step % refresh_target_network_freq == 0:\n",
    "            # Load agent weights into target_network\n",
    "            target_network.load_state_dict(agent.state_dict())\n",
    "\n",
    "        if step % eval_freq == 0:\n",
    "            # eval the agent\n",
    "            mean_rw_history.append(evaluate(\n",
    "                make_env(ENV_NAME, seed=step), agent, n_games=3, greedy=True, t_max=1000)\n",
    "            )\n",
    "\n",
    "            clear_output(True)\n",
    "            print(\"buffer size = %i\" %\n",
    "                  (len(exp_replay)))\n",
    "\n",
    "            plt.figure(figsize=[16, 5])\n",
    "            plt.subplot(1, 2, 1)\n",
    "            plt.title(\"Mean reward per episode\")\n",
    "            plt.plot(mean_rw_history)\n",
    "            plt.grid()\n",
    "\n",
    "            assert not np.isnan(td_loss_history[-1])\n",
    "            plt.subplot(1, 2, 2)\n",
    "            plt.title(\"TD loss history (smoothened)\")\n",
    "            plt.plot(smoothen(td_loss_history))\n",
    "            plt.grid()\n",
    "\n",
    "            plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "buffer size = 10000\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6UAAAE/CAYAAAC+Q2VKAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAByt0lEQVR4nO3dd3xb1f3/8ddH3jPLiRPH2YsMCJCwEkZYBVpWW/iVFijQQQfdm7bftrSl306+7bct/ZZOKLSFMsosG7MJJCEhO3G2E8eOnWF5W9L5/aFrx0k8ZEeyZOn9fDx4WLq6uvqcq6Crj845n2POOURERERERETiwRfvAERERERERCR1KSkVERERERGRuFFSKiIiIiIiInGjpFRERERERETiRkmpiIiIiIiIxI2SUhEREREREYkbJaUig4iZlZnZx+IdR1+Z2f+Z2X9F+ZjXm9kr0TymiIhEZiA/g81sopk5M0vv5vFvmtkfByKWvjKzT5jZL+MdR08S5buFmX3PzO72bheb2Vozy4p3XDIwlJRK3JnZVjNrNbOiw7Yv9y5CE+MUmkSJc+6TzrkfxDsOEZFUZGb1nf4LmVlTp/tXe8lAm5n5vf82mNlvzGxMvGOPhHPuR865XpOqgU6+zCwT+Dbws4F6zd50TvwSmXOuCngBuDHescjAUFIqiWIL8MH2O2Z2LJATv3AO6u6X2Ri/pplZ3P7/jEebRUQkNpxz+e3/AduBSzptu8fb7V7nXAEwHHgvMBpYOlgS01jr53X5MmCdc25nLGJKAfcAn4h3EDIwlJRKovgb8OFO968D7uq8g5llmdnPzWy7mVV5Q0JzvMeGmdljZrbHzPZ5t0s7PbfMzH5gZq96vwI/fXjPbKd9F5lZhZl93cx2A38xM5+ZfcPMNplZrZndZ2bDvf3vNLMve7fHer27n/buTzWzvd7FLJIYbzWzV4FGYLKZnW9m68zsgJn9BrDuTqD36+f9Znav18ZlZja30+MlZvaA9/pbzOxzXTz3bjOrA67v4vg9nf/2c/ZNM6vxer+v7vTcv5rZD73bRV7b93vn5uX2C72ZzfTOw34zW21ml3Y6xggze8TM6szsTWDKYfEdY2bPeMdcb2b/r7tzJSIiXXPOtTnnVgMfAPYAX47keWa2wMze8q5Xb5nZgk6PXW9mm71r05b264N3jXzRe06Nmd3by8tc7V2DaszsW52O33nYZ7Z3Lav1riVvWXgo6K3AGcBvLNxD/JsI4j78uvxlM1t6WLu/bGb/7ibei4AXO+3bZWydXuuHZvaaF9+j3nXvHu+695Z1GjnWS9wl3vVyr5mVm9nHve0XAt8EPuC9xopOsU6wbr4jmdmpXlz7zWyFmS067Bx1+/2ql+dO8t5/v5k9Axz+vWwx4e9CE7o5v5JElJRKongDKPSSkjTCF8PDh5f8BJgOHA9MBcYC3/Ee8wF/ASYA44Em4DeHPf9DwA3AKCAT+EoP8Ywm/GvxBMJDRz4HXA6cBZQA+4Dfevu+CCzybp8FbPb+ApwJvOyccxHGeK33egXAAeABwkN/ioBNwMIeYobwr7L/8mL/O/BvM8vwkr5HgRWEz9u5wBfM7ILDnns/MJTwr5OH6+n8Q/icFXnbrwPuMLMZXRzny0AFMBIoJnyBdGaW4cX4NOH36LPAPZ2O8VugGRgDfMT7DwAzywOe8do8inCv++1mNrvbMyUiIt1yzgWBhwkncj2y8I+0jwP/C4wAbgMe95KqPG/7RV5P7AJguffUHxD+zB8GlAK/7uWlTgdmEL6GfcfMZnaxz3XAEGCcF8sngSbn3LeAl4HPeD3En+kp7k7H63xd/l9g0mGvew3hH9a7ciywvrfYOj1+lfd6Ywn/8Po64e8Nw4G1wHeh5/PtHecfhK+zJcAVwI/M7Fzn3JPAjwj3iuc75zp+uKab70hmNtZ7rR96cXwFeMDMRkbhuX8HlhL+7vAD7/x0cM4FgHKgc5ySpJSUSiJp7y09H1gHdAx3MTMDPg580Tm31znnJ/zBehWAc67WOfeAc67Re+xWDiaG7f7inNvgnGsC7iOcXHUnBHzXOdfi7f8J4FvOuQrnXAvwPeAKCw9zfRE4w0v8zgR+ysHk8Szv8Uhj/KtzbrX3QXwRsMY5d79zrg34JbC7l3O4tNP+twHZwKnAScBI59z3nXOtzrnNwB/az5/ndefcv51zIa/NHXo7/538l3fOXiR8Ieqqt7KNcGI5wftFvj1pPxXIB37sxfg88BjwQe+HivcD33HONTjnVgF3djrmxcBW59xfnHMB59wywgn9Fb2cLxER6d4uwslEb94DbHTO/c37DP4H4ev4Jd7jIWCOmeU45yq9nlgIXw8mACXOuWbnXG+Fk25xzjU551YQ/pG1q2SljXCiNtU5F3TOLXXO1fUzbuh0Xfau//cSTkTxfvicSPha1ZWhgL8Psf3FObfJOXcA+A+wyTn3rPed4F/ACb3FbWbjCCfvX/fO6XLgj4ST3Z509x3pGuAJ59wT3veDZ4AlwLuP5rlmNp7wd5P27w0vEf5h+nB+7zxKklNSKonkb4R/bbuew4buEu5VyyU8v2W/me0HnvS2Y2a5ZvZ7M9tm4eGnLwFDvWSmXeeErpFwAtSdPc655k73JwAPdXrttUAQKHbObQLqCX8In0H44rTL6+HrSEojjHFHp9slne97iVvnx7vSef8QB38pnQCUtMfvteGbhHsqu3rtw/V4/j37nHMNne5v8177cD8j/Mvn0xYezvUNb3sJsMOLu/Mxxnqvk35YjNs63Z4AnHJY+64m3HsrIiL9MxbYG8F+JRz6mYx3f6x3XfgA4V7BSjN73MyO8fb5GuFpKW9aeMrGR+hZJNfxvwFPAf80s11m9lNvJE6f4u50//Br453Ah7wfa68F7vOS1a7sI9zDGmlsVZ1uN3Vxv729PcVdArT/eNxdm7rS3bmdAFx52PX1dMI/Lh/Nc0vo+nvD4QqA/b3ELklASakkDOfcNsIFj94NPHjYwzWEP5BnO+eGev8NceGiDRAeEjoDOMU5V0i4xxJ6mIPZWziH3d9BeOjR0E7/ZbuDxQteJNwrl+lte5Fwr+8wDg5TiiTGzq9bSXiIT3in8AVwHD3rvL+P8HCoXV78Ww6Lv8A51/mXzsPb3Flv5x9gmDdMq91477UP4ZzzO+e+7JybTPjX6C+Z2bnevuPs0EIS4wn3mO8BAoe1f3yn2zuAFw9rX75z7lM9tElERLrhfRZfQnjIa292EU5AOmv//MY595Rz7nzCycg6wiN1cM7tds593DlXQnhE0u1mNvVo4vZG4NzinJtFeKjwxRysWXH4da7HuLt6jnPuDaCV8I/QH6L7obsA7xCe9hJJbH3RU9y7gOFmVtDFY9Dztb4rO4C/HXZ9zXPO/fgon1tJ198bOnij0aYS7hWXJKekVBLNR4FzDvvlrL3X7w/A/5jZKOgoKtQ+J7KAcNK035tr8d0ox/V/wK3tk+3NbKSZXdbp8ReBzxDu/QQoIzwn8hUXnpfTnxgfB2ab2fu8D+bP0XvP37xO+38BaCE8X/dNoM7CxZtyzCzNzOaY2Um9tpyIzn+7W8ws08zOIHyx/dfhxzKziy1c3MKAOsI9zkHCBQ0agK9582AXEf5C9E/vHD4IfM/rcZ7FoXNPHgOmm9m13nMzzOykbuYbiYhIN7zPz5mE5yWOJjwVpDdPEP4M/pCZpZvZB4BZwGMWLjJ0qZd8tBAeWRT0XutKO1jwbx/hhCnYxfH7Ev/ZZnasNwqpjvCQ2fZjVgGTI4m7l5e5i3BNiEAvQ46foNM0nV5i64tu43bO7QBeA/7bwoWVjiP83aq9VkQVMNEiryR8N+FhwRd43x2yLVzcsLTXZ/bwXK8jYgkHvzeczqHDpgFOJjw1p6seVEkySkoloXhzKZZ08/DXCQ/7fMMb/vos4Z5HCM+3zCHco/cG4aGl0fQr4BHCQ0793muc0unxFwknne1J6SuEh7u+1GmfPsXonKsBrgR+DNQC04BXe4nzYcLDpPYRHlb0Pu+X2SDhD/vjCfdG1xCeYzKkl+N11tP5h/DwnX2Ef6W9B/ikc25dF8eZ5j23nnARh9udc2XOuVbgUsJzaWuA24EPdzrGZwgPCdoN/JVw8Qcg3PsKvIvwHNdd3j4/AbTotohIZD5gZvWEh0o+Qvi6M885d8SIl8M552oJ/xD5Ze95XwMu9q5jPm/7LsJDgc8CPu099SRgsfe6jwCfd85tOcp2jCZctK+O8FSbFzlYOPFXhOtB7DOz/+0l7p78DZhDz72kEJ4jeYyZtU9l6Sm2iEUQ9wcJz3XdBTxEuEbGM95j7T8W15rZsgheawfhQojfJDxqaQfwVSLIISJ47ocIf5faS/iH+sOnbl1NuFNAUoCFp6mJyGBnZt8jXDzhmji89iLgbudcJL+cioiIDFoWXg6tGjjRObexl31vBGY5574wELElC29U1ovACYfV+JAklR7vAEREREREBpFPAW/1lpACOOfuGIB4ko5zrhrQFJwUoqRURERERCQCZraVcIHCy+MbiUhy0fBdERERERERiRsVOhIREREREZG4UVIqIiIiIiIicZMQc0qLiorcxIkTo3KshoYG8vLyet8xiaRimyE1252KbYbUbHcqthmi1+6lS5fWOOdGRiGklKVr88DSOYqMzlNkdJ56p3MUmWiep56uzQmRlE6cOJElS7pbmrJvysrKWLRoUVSONVikYpshNdudim2G1Gx3KrYZotduM9Ni60dJ1+aBpXMUGZ2nyOg89U7nKDLRPE89XZs1fFdERCSJmNmfzazazFZ12vYzM1tnZu+Y2UNmNjSOIYqIiBxCSamIiEhy+Stw4WHbngHmOOeOAzYANw90UCIiIt1RUioiIpJEnHMvAXsP2/a0cy7g3X0DKB3wwERERLqhpFRERCS1fAT4T7yDEBERaZcQhY5EREQk9szsW0AAuKebx28EbgQoLi6mrKwsKq9bX18ftWMlK52jyOg8RUbnqXc6R5EZqPOkpFRERCQFmNl1wMXAuc4519U+zrk7gDsA5s+f76JVcVFVLnuncxQZnafI6Dz1TucoMgN1npSUioiIJDkzuxD4OnCWc64x3vGIiIh0pjmlIiIiScTM/gG8Dswwswoz+yjwG6AAeMbMlpvZ/8U1SBERkU7UUyoiIpJEnHMf7GLznwY8EBERkQipp1RERHpVeaCJ9bv98Q5DRERSREsgyMsb9xAKdTkFXpKMklIREenVz55cz01/XxbvMEREJEXcdM/bXPunN/n7m9vjHYoMACWlIiLSq5qGVmrqW+IdhoiIpIDWQIhn11YB8NsXyummYLgkESWlIiLSK39zG3VNbfpiICIiMdccCAIwdVQ+lQea2VqrouHJTkmpiIj0yt8cIOSgoTUY71BERCTJNbeFrzWnTR4BwNvb98UzHBkASkpFRKRX/uY2AOqa2uIciYiIJLuWthAAc8YWkpORxpfuW8G63XVxjkpiSUmpiIj0qr45AEBds5JSERGBP768mZNvfTYm1XFbvOG7uZnpfPDk8QB89+HVmkKSxJSUiohIj4Ih1zFst64pEOdoREQkEfzw8bVU+1tYFoOhtc1eT2lWuo/vXDKLWy6dzeIte7n7jW1Rfy1JDEpKRUSkR+29pKDhuyIiEnbi+KEAPPT2Tp5dU8UfXtrMC+uq+fr97/DW1r1c9+c32bG3fwWK2ueUZmekAXDNqRM4fWoRP396g3pLk1R6vAMQEZHE5m85mIhq+K6IiMDB3sx7Fm/nnsWHriV639IdOAcPL9/JieOHsWNfI9kZaVx2/NiIjt0SCB+7PSlN8xnnzhzFK+U11NS3MrIgK4otkUTQa1JqZjOAezttmgx8B7jL2z4R2Ar8P+fcPu85NwMfBYLA55xzT0U1ahERGTB+9ZSKiMhh9ja0cuL4oazaWcfMkkLGDcuhdFguW2rq2VbbSGNrkJ8/veGQ58ybMIzSYbm9Hru9pzQr/eCgzskj8wHYvKdeSWkS6jUpdc6tB44HMLM0YCfwEPAN4Dnn3I/N7Bve/a+b2SzgKmA2UAI8a2bTnXNaR0BEZBA6JClt1pxSEZFU55xjb0Mrl58wlj9edxJDczLw+azjMYCfPLme/3txE19513TmjhvKtX96k4eX7+Kms6f2evz2Xtj2nlKAyUV5AGypaeAUb6kYSR59Hb57LrDJObfNzC4DFnnb7wTKgK8DlwH/dM61AFvMrBw4GXg9KhGLiMiA8ncasqueUhER8bcEaA2GGJGXyfC8zEMeMwsnp188fxqXzi1hVkkhACdPHM59S3bwybOmkOYlsN1pr76bnXGwp7RkaA6Z6T421zREsymSIPpa6Ogq4B/e7WLnXCWA93eUt30ssKPTcyq8bSIiMgjVt3TuKVVSKiKSqpxzBEOODbv9AEckpJ1lpad1JKQA1y2YyLbaRi7+9StcdUfPfVUHq+8e7ClN8xlzS4fw+DuVHcN7JXlE3FNqZpnApcDNve3axbYjymSZ2Y3AjQDFxcWUlZVFGkqP6uvro3aswSIV2wyp2e5UbDOkZrsTqc1Lt4cT0bwM2LSjkrKy6Jf/b5dI7RYRkYPW7/ZzwS9fOmTb8Pzuk9LDXTC7mJEFWaytrAOg2t/MqILsLvc9WH330P6zL54/nQ/9YTH/WlrBtadO6Ev4kuD6Mnz3ImCZc67Ku19lZmOcc5VmNgao9rZXAOM6Pa8U2HX4wZxzdwB3AMyfP98tWrSor7F3qaysjGgda7BIxTZDarY7FdsMqdnuRGrzmrJyWLOeiSMLycrJYNGiU2P2WonUbhEROeiV8pojthV3k1R2JT3Nxy8/cDxX/3ExACt2HOD8WeHnb6tt4P6lFXzxvOn4fHZE9d12C6YUcczoAu5XUpp0+jJ894McHLoL8AhwnXf7OuDhTtuvMrMsM5sETAPePNpARUQkPuqbA6T7jFEFWRq+KyKSojbvqQfg2lMnsO4HF3LPx05h5piCPh1j4dQi1n7/QgA+ftcS7lm8DeccX75vBb9+vpw1Xi9qV9V3273/xFJW7NjP9tr+rYEqiSmipNTMcoHzgQc7bf4xcL6ZbfQe+zGAc241cB+wBngSuEmVd0VEBi9/c4CC7HSG5GRQ16TquyIiqWj9bj8nTxzODy6fQ3ZGGgunFnUUNeqLnMw0LpozGoBvPbSKdyoOcMArord8x34AmgNBMtN9XR5/0YyRALy++cieWxm8IkpKnXONzrkRzrkDnbbVOufOdc5N8/7u7fTYrc65Kc65Gc65/8QicBERGRj+5jbys9MpzMlQT6mISAp6ZWMNS7btY8bovvWMdud318xjybfPA+C2ZzawsTrcC7t0W7hmQUtbiOwuekkBpo7KZ0ReJos37+3ycRmc+rokjIiIpBh/c4CCrAwKszOoa2rDOdevX8dFRGTwqapr5lN3L2VSUR7XLZgYteMW5WcxJCeDFzfs6dj27+U7SfMZIeeOmE/azsw4dcoIXtq4h7ZgiIy0vi4mIolI76KIiPTI3xIevluYk07IQUOrZmSIiKSKv72+jca2IH++/iSmjsqP6rG/d+ksTp40nHkThvHwTQu54sRS7l9awWMrKsnK6D5Ned8JY6mpb+XZNVXd7iODi3pKRUSkR/7mAGOHZlOYnQFAXVMb+Vm6fIiIJCPnHM+treb0aUVkZ6TxSnkNx48byqSivKi/1ntPKOW9J5R23J8xuoAHllXQGgyRnd51TynAohmjGJGXyXPrqrno2DFRj0sGnnpKRUSkR/7mNgqyMyjM8ZJSzSsVEUlar2+q5WN3LeG2ZzZwoLGNdyr2s2DKiAF57eyMNCaOyOu43Z00nzGpKI8de1WBN1koKRURkR7Vtw/f7egpVQVeEZFk9ezaagDueGkzc7//NCEHZ04fOWCvP9Hrkc3pISkFGDc8l4p9TQMRkgwAJaUiItIt51zHkjCFOeEhu3VN6ikVEUlGoZDjqdW7j9g+b/ywAYvhGK/C7zWnTehxv9JhOVQeaKItGBqIsCTGNClIRES61dQWJBhy5HvVd0HDd0VEktUbm2vZub+J/7p4Fuk+45EVu7h0bgk+38BVXP/02VO5cM5ojisd2uN+44blEnJQub+Z8SNyByY4iRn1lIqISLf8zeGhugXeOqWgnlIRkWSxY28jH/3rW1TVNQNw/9IKCrLTufqU8Vy3YCIPfGpBVJeBiUR+VnqvCSlA6fAcAM76+Qsc0HVp0FNSKiIi3eqclBZke8N3mzWnVEQkGfxraQXPravmWw+twt/cxhOrKrlkbkmPRYYSxXGlQ5k7bijOwdxbnubRFbviHZIcBSWlIiLSLb83VLcwO4OMNB+5mWnqKRURSRIrK/YD8ML6au5bUkFzW4gr5pX2/KQEkZ+VzsM3LeQL500D4BmtWTqoKSkVEZFutfeU5nu9pIXZGZpTKiKSBIIhx5Kt+5hclEcw5PjBY2uYPDKPE8YNjXdoffKF86ZzxrQiNtfUxzsUOQpKSkVEpFudh+8CFOaka0kYEZEksGZXHf6WAJ9aNKVj2/+bPw6zgStqFC1TRuazZU8Dzrl4hyL9pOq7IiLSrfqWcK9ogVd5Vz2lIiLJYfGWWgDOmDaS714yi8bWIB9ZOCnOUfXPlJF5NLQGqaprYfSQ7HiHI/2gpFRERLrVMXw3q72nNINqf3M8QxIRkShYvGUvE0bkMnpINjcM0mS03ZRR+QAs2baXi48riXM00h8avisiIt2qOzwpzdbwXRGRZLCy4sCgmz/anZMmDmfqqHx+8fQGAsFQvMORflBSKiIi3apvDpCflU6at3B6YY6G74qIDHZ7G1rZXdfMrJLCeIcSFRlpPr7yrhlsqWngqdWqwjsYKSkVEZFu+ZvbOoocgTentKlNxSRERAaxtZV1AMwckxxJKcD5s4qZMCKXP72yOd6hSD8oKRURkW75vZ7SdoU56YQcNLQG4xiViIj010sb9vDShj1AciWlaT7jhgUTWbZ9P29u2RvvcKSPlJSKiEi3/C1H9pQC1DVpCG+iMrM/m1m1ma3qtG24mT1jZhu9v8PiGaOIxEdTa5AP//lNfv/SZsYOzaEoPyveIUXVlfPHMaogi4/89S3Kq7Vu6WCipFRERLpV3xzoWA4GwnNKAc0rTWx/BS48bNs3gOecc9OA57z7IpJiOidqJ01Mvt+m8rLSefDTCzDgR0+sjXc40gdKSkVEpFv+5gD5XfaUqgJvonLOvQQcPnbtMuBO7/adwOUDGZOIJIb1Vf6O2+NH5MUxktgpHZbLdQsm8vy6ag5oVM+goaRURES6VdccoDD70DmlgC70g0+xc64SwPs7Ks7xiEgcrN8dLnC0aMZIrjl1fJyjiZ0FU0YAcPpPnmf5jv3xDUYikt77LiIikqrqW9oOHb6rOaVJzcxuBG4EKC4upqysLCrHra+vj9qxkpXOUWR0niJz+Hmqb3X8clkz5ftDjC/wcf2kRtYsfYM18QsxppoD4Qrx/uYAn/rra/z3GblH7KN/S5EZqPOkpFRERLrUFgzR3Bai4JDqu5pTOkhVmdkY51ylmY0BqrvayTl3B3AHwPz5892iRYui8uJlZWVE61jJSucoMjpPkTn8PD23tory55ewYMoIvvnumcwZOyR+wQ2QY9e+wsqdB6hscEyccxITiw4drqx/S5EZqPOk4bsiItIlf3N43mjnOaXtlXg1p3TQeQS4zrt9HfBwHGMRkQHWXuDod1fPS4mEFOCBTy3gha8sAuD5dV3+DicJJKKk1MyGmtn9ZrbOzNaa2Wk9lZc3s5vNrNzM1pvZBbELX0REYsXv9YZ2Hr6bkeYjNzNNPaUJzMz+AbwOzDCzCjP7KPBj4Hwz2wic790XkRRRXl1PUX4WQ3Izet85SWSm+5hUlMeUkXm8sF5JaaKLtKf0V8CTzrljgLnAWropL29ms4CrgNmES9LfbmZp0Q5cRERiq72ntPM6pQBDcjI0pzSBOec+6Jwb45zLcM6VOuf+5Jyrdc6d65yb5v3VyvIiKaR8Tz1TRyVntd3evGv2aF7bVMv22sZ4hyI96DUpNbNC4EzgTwDOuVbn3H66Ly9/GfBP51yLc24LUA6cHN2wRUQk1jqS0qxDk9LC7Az1lIqIDBIrKw6welcdx4wujHcocXH9gomkmfGX17bEOxTpQSQ9pZOBPcBfzOxtM/ujmeXRfXn5scCOTs+v8LaJiMgg0tXwXQgvC6M5pSIiie9AUxufvHspI/Oz+PSiKfEOJy6KC7M5ZfJwFm/WAJFEFkn13XTgROCzzrnFZvYrvKG63bAutrkjdlLZ+ahJxTZDarY7FdsMqdnuRGjzWzvDSemaFUupLT/4G2ZrQzPVLS4m8SVCu0VEksUdL21i14EmHvzUAkYVZsc7nLg5ftxQbi/bRFNrkJxMzSpMRJEkpRVAhXNusXf/fsJJaXfl5SuAcZ2eXwrsOvygKjsfPanYZkjNdqdimyE1250Ibd722lZYuZrzzlrIiPysju0PVy1nyba9MYkvEdotIpIMbrpnGY+vrOT8WcWcMH5Y709IYnNLhxIMOVZU7OfUySPiHY50odfhu8653cAOM5vhbToXWEP35eUfAa4ysywzmwRMA96MatQiIhJz7cN387MPn1Oq4bsiIomstinE4ysrAfjcOdPiHE38zZ84jILsdL583wqa24LxDke6EGn13c8C95jZO8DxwI/opry8c241cB/hxPVJ4CbnnN59EZFBxt8cIDPdR1b6oUOdCnMy8De3EQodMTNDREQSwIo94a/ez37pLI4tTY11SXsyNDeTH14+h537m9hYVR/vcKQLkQzfxTm3HJjfxUPndrP/rcCt/Q9LRETizd8SoDD7yMtEYXYGIQcNrYEjiiCJiEh8rdtdx0PlrcwoLmDKyNRcBqYrM0YXALC1tkGJegKKtKdURERSjL85QH5WF0lpTnhbXbOG8IqIJJKGlgD/7/9eJ82M311zImZd1R9NTeOH5wKwrbYhzpFIV5SUiohIl/zNbV32hBZ62+qatFapiEgi2VLTQF1zgKtnZjJ5ZH68w0kouZnpFBdmsbW2Md6hSBeUlIqISJf8zQEKuhq+m6OkVEQkEVXsCydco3LVQ9qVCcPz2K6kNCEpKRURkS7Vd5eUtveUaviuiEhCqdjXBEBRjr7id2VacT5rKusIBEPxDkUOo3+xIiLSJX9zG/lZXQzfbZ9Tqp5SEZGEsmNvIwVZ6eRGVMo09Zw2ZQT1LQFW7jwQ71DkMEpKRUSkS90O3+3oKVVSKiKSSCr2NTF2WI4KHHXj1MkjAHh4+S6c07JmiURJqYiIHCEUctS3dr0kTHuiWtek4bsiIonira17eX1zLVNGqcBRd4rys7js+BL++tpW1tRqCG8iUVIqIiJHaGgN4Bzkd5GUpqf5yMtMU0+piEgC+dETaxmWm8m33j0z3qEktO9fOgeAHX4lpYlESamIiBzB7xUx6mpJGAhX4NWcUhGRxLBmVx1vb9/Px86YRMnQnHiHk9CG5GZQkJ3OniYlpYlESamIiBzhYFLadbWMwuwM9ZSKiCSIdyr2A3DezOL4BjJIjB+ey54mzSlNJEpKRUTkCPUt4YSz+57SdM0pFRFJEFtqG8hM86mXNELjhuWyp1E9pYlESamIiByhfQ3S/Cz1lIqIJLptNY2MG55Dmk9VdyMxbngOlQ2Oz/x9WbxDEY+SUhEROUL78N2uqu+CN6dUSamISNz98eXNPLl6NxNH5MU7lEHjrOmjAHjsnUqWbN0b52gElJSKiEgX6nsrdJSt4bsiIongh4+vBSC3m5EtcqTTpxXx+/NyKcxO5+9vbo93OIKSUhER6YLf6wXtakkYCPeU+pvbCIVUKEJEJF6q65o7bl932oQ4RjL4ZKUb580s5qG3d7Jzf1O8w0l5SkpFROQI/uYAPoO8zLQuHy/MziDkwuuZiohIfKzceQCAf33yNOZPHB7naAafd80uxjlY+OPnub2sPN7hpDQlpSIicgR/cxv5WemYdV00ozAn3IPaXhBJREQG3ptb9+IzmDWmMN6hDErnzxrNr646ntMmj+B3ZZtoCQTjHVLKUlIqIiJH8LcEup1PCuGeUoC6JhU7EhGJh2DI8fDbu1g0YxR5mk/aL2k+47Ljx3LjWZPxNwf4z8rd8Q4pZSkpFRGRI/ibAxR0M58UwnNKQUmpiEi8vLllL7vrmrliXmm8Qxn0Tp9axPTifL7yrxVsq22IdzgpSUmpiIgcwd/c1nNS2t5TquG7IiJx8fqmGtJ8xhnTiuIdyqCXkebj1x88kUDI8fb2/fEOJyUpKRURkSPU9zZ8t31OqXpKRUTi4vXNtcwZO6THz2qJ3KSiPNJ9xoYqf7xDSUlKSkVE5Ai9Dt/t6ClVUioiMpCcc2zaU8/yHfs5bfKIeIeTNDLTfUwqymNDVT0V+xoJBEPxDimlKCkVEZEj+JsD5PdQOKM9Ya1r0vBdEZGBdPfi7Zz7ixdpCzqumDc23uEklenFBTy7torTf/IC9y+tiHc4KUVJqYiIHMI5580p7X5IWHqaj7zMNPWUiogMsKdWhSvEvvvY0UwdVRDnaJLLJXNLGDMkG4B3vDVgZWAoKRURkUO0BEK0BV2Pw3chXIFXc0pFRAZOMORYUbGfq04ax+1Xz4t3OEnnwjmjef3mc5kztpCd+5riHU5KiSgpNbOtZrbSzJab2RJv23Aze8bMNnp/h3Xa/2YzKzez9WZ2QayCFxGR6PN7FXV7TUqzM9RTKiIygJZs3Yu/OcDJk4bHO5SkNqkony01WhpmIPWlp/Rs59zxzrn53v1vAM8556YBz3n3MbNZwFXAbOBC4HYzS4tizCIiEkN+L9Hsvac0XXNKRUQGiHOO257ZQFF+FhfOGR3vcJLapBG5bN/byN6G1niHkjKOZvjuZcCd3u07gcs7bf+nc67FObcFKAdOPorXERGRAdTRU5rV8zIDQ3LUUyoiMlBeKa9h8Za9fPacqeRm9vyjoRyd6aPDc3Xf9T8vEQq5OEeTGiJNSh3wtJktNbMbvW3FzrlKAO/vKG/7WGBHp+dWeNtERGQQqG/R8F0RkUTzh5e3MGZINledPC7eoSS9C2eP5voFE6mpb+GtrXvjHU5KiPRnloXOuV1mNgp4xszW9bCvdbHtiJ8YvOT2RoDi4mLKysoiDKVn9fX1UTvWYJGKbYbUbHcqthlSs93xbPOS3eGkdN2q5TRt7372RV1tC7V1gajGmYrvtYhIbw40tfFaeQ0fPWMSWemaFRdr6Wk+vnrBDO59awdPrKzkFK0HG3MRJaXOuV3e32oze4jwcNwqMxvjnKs0szFAtbd7BdD5J5xSYFcXx7wDuANg/vz5btGiRf1uRGdlZWVE61iDRSq2GVKz3anYZkjNdsezzdVLdsDydzjn9NMYNzy32/2Wta7nuR3lnHnmWfh8Xf0e2Xep+F4PJDP7IvAxwj8WrwRucM41xzcqEenN317fSiDkuGC25pIOlLysdE6cMJSl2/fFO5SU0OvwXTPLM7OC9tvAu4BVwCPAdd5u1wEPe7cfAa4ysywzmwRMA96MduAiIhIb9ZFW383JwDmob1Wxo8HAzMYCnwPmO+fmAGmECxOKSAJbsWM/P396A+85dgwnjBsa73BSyrFjh7J+t5/mtmC8Q0l6kfSUFgMPmVn7/n93zj1pZm8B95nZR4HtwJUAzrnVZnYfsAYIADc55/ROiogMEu2FjvKyep9TClDX1NZxWxJeOpBjZm1ALl2MZBKRxPKLZzYwPC+TH7//WLzv4zJA5pYOoS3oWL/bz1z9IBBTvSalzrnNwNwuttcC53bznFuBW486OhERGXD+5jZyMtLISOt5ME1hTvgSUtcUgGE97ioJwDm308x+TviH5Cbgaefc03EOS0R60BYM8cbmWj586gQK9OPfgJszdggAayrrlJTGmOpJi4jIIfzNgV6H7kKnnlJV4B0UzGwY4WXbJgH7gX+Z2TXOubs77aMihHGicxSZVDtPFf4QrYEQvrpdlJVV9/4ET6qdp/6I5ByFnCPdBy+9vY4xjZsHJrAEM1D/lpSUiojIIepbIkxKcw4O35VB4Txgi3NuD4CZPQgsADqSUhUhjB+do8ik2nl66O0KYAVXnHsK04sLIn5eqp2n/oj0HE18+0WCOXksWjQ/9kEloIH6txTpOqUiIpIi6prbyI9gmNjBnlIVOhoktgOnmlmuhSemnQusjXNMItINf3Mbf35lK5npPiYX5cU7nJQ1cUQe22ob4x1G0lNSKiIih/A3ByiMqKe0fU6pekoHA+fcYuB+YBnh5WB8eL2iIpJ4fvNCOat3HeCr75pBei9z/CV2JhXlsrW2gVDIxTuUpKZ/4SIicohIh+/me9V5Nad08HDOfdc5d4xzbo5z7lrnXEu8YxKRI/3i6fX8/sXNXHxcCR8/c3K8w0lpE0bk0RIIsbtOSzrHkpJSERE5hL+5rSPh7El6mo/8rPRw9V0REYmaB5ZWkJ+VzrffMzPeoaS8Sd7Q6a01DXGOJLkpKRURkUOEq+9GtvRAYXa6ekpFRKIoEAxR5W/hhoUTGVWYHe9wUt5ELyndUqukNJaUlIqISIdAMERjazCi4bsQrsCrOaUiItFT7W8hGHKMGZIT71AEGFOYTWa6T8WOYkxJqYiIdGhoCQL0oac0Qz2lIiJRtGt/EwAlQ9VLmgh8PmPC8Fy29GP47mf+vowb71oSg6iSj9YpFRGRDu0JZkEEc0ohXIF3134VfxARiZadHUmpekoTxcSivD4npdX+Zh57pzJGESUf9ZSKiEgHv7fmaMTDd9VTKiISVZUHwj/0jRmintJEMW1UPltrGmgJBCN+zvNrqztuO6flZHqjpFRERDr423tKIx2+qzmlIiJRtXhzLaMKsiL+HJbYmzG6gEDI9am3tKH1YAKrKvW9U1IqIiId6lvCF878iHtK0/G3BLSouIhIFGytaeCF9Xv44Mnj4x2KdDK9uACA9bv9ET+nNRDquF3ToCWhe6OkVEREOvR5+G5OBs5Bfat+BRYROVovb9wDwBXzSuMciXQ2eWQeaT7jB4+tiXi90rbgwaS0tr41VqElDSWlIiLS4eDw3cjnlAIawisiEgVv79hPUX4WpcNU5CiRZKWn8bULZtDSFuLDf37zkF7Q7nROSveqp7RXSkpFRKSD3xu+WxjxnNJw8qr5MiIiR2/59v2cMH4oZhbvUOQwnzhrCre+71i2721k5c79ve7f2ikprVFPaa+UlIqISAd/c4B0n5GVHtnloaOnVBV4RUSOyv7GVjbXNHD8uKHxDkW6sXDKCADe2Ly3131bAyFyMtIA+Pa/V3HKj57lhr+8SbW/f8uohUKuo/d1S00Dd72+ladX746o13YwUFIqIiId/M1tFGSnR/wrfWGOhu+KiETD8h37AThh/NC4xiHdG5GfxfTifBZv6T0pbQuGyM1MY1JRHgBF+Vm8vrmWc3/+Il+//x2C3RQIfG1TDRf8z0s8uKyiY9vO/U0s+nkZZ/70Bd6p2M97b3+V7zy8mhv/tpTHV+6KTuPiTEmpiIh08DcH+rQMwcGeUg3fFRE5Gm9v34/P4LjSofEORXowt3Qoa3bV9bpfW8CRkebjsc+ezuOfO51HPhP+b1ZJIfcu2cH5t73I2sojj/PI8l2sr/LzlX+t4NXyGpxz/OGlzWzf20jlgWYu/c2r5GSk8fePnwIkz/QZJaUiItKhvjlAflZkRY6g85xS9ZSKiPTXw8t38qvnNjK9uKBPn8Ey8GaMLqCmvoXa+p6LF7UGQ2SkG3lZ6cwuGUKaz5heXMA/bzyVd80qZnNNA795vvyI5725dS8njB/KiPwsrv7jYk669Tn++tpWzp9V3LHPX244qWOYd0sgeMQxBiP9qxcRkQ7hntLILw3tX540p1REpP/+s3I3AN+7dHacI5HeHDO6EAivWbpgala3+7UGQ2SmHdn/Z2bc8eH53PLoau55Yzs19S38/sVNjB2awxnTR7J5TwPfuOgYrjppHM+urealDXsozEnn8+dOZ8LwXLIz0jhmdCEBb35pS1tyzClVUioiIh3qmtsoHZYb8f7paT7ys9KTZviQiMhAc86xdPs+3nvCWE6dPCLe4Ugvpo/OB2Ddbj8LphZ1u19bIERGF0lpuyvmlfKXV7fy7YdW8eTq3R3b8zLTeM+xYxiam8kV80oPWbP22xfP6ridnuYj3We0qNCRiIgkm/qWAIV96CkFKMxOV0+piEg/baiqZ4+/hRMnDIt3KBKBkflZjBuew0sb9/S4X2swRGYPlexnlwzhmNEFHQnp/11zIh88eRx/+PB8xg2P7MfhrHRf0gzfVVIqIiId/M0B8vualOZkaE6piEgftQSCPLOmio/d9RZDcjI4b+aoeIckETAz3n3sGF7ZWMO+hu7XH20L9txTCvCF86YDcOHs0Vw4Zwz//b7jeux9PVxWRpp6SkVEJLk456hv6ducUghX4FVPqYhI39zzxnY+ftcS9vhb+MsNJzFmSE68Q5IIXTh7NIGQ47VNtd3u0xZwXc4pPeQ4c0bz1rfO42dXHtevOLLSfUkzpzTipNTM0szsbTN7zLs/3MyeMbON3t9hnfa92czKzWy9mV0Qi8BFRCS6GluDBEOuT0vCQLgCr+aUioj0Tfu6pE9+/kxOHK+hu4PJ7JIhZKb7WL5jX7f7hKvv9p5qjSzI6vN1t12qDt/9PLC20/1vAM8556YBz3n3MbNZwFXAbOBC4HYzS4tOuCIiEiv1LeHEUj2lIiKxt6ayjvNmFjOxKC/eoUgfZab7mFFcwB9e3sJvnt/Y5T6tgRCZaRbTOLLSU2z4rpmVAu8B/thp82XAnd7tO4HLO23/p3OuxTm3BSgHTo5KtCIiEjN+L7Hs6xp5mlMqItI3Ta1BNu+pZ1ZJYbxDkX46cfxQAH7+9AaaWo/srWzrpdBRNGRl+GhuS62e0l8CXwM6p+LFzrlKAO9v++zsscCOTvtVeNtERCSB1TWHe0oL+zx8NwN/S4BQyMUiLBGRpLNq1wFCDuYoKR20vn7RMXz7PTMBeGTFziMej6TQ0dEKD99Njp7SXn8ON7OLgWrn3FIzWxTBMbvqpz7im4qZ3QjcCFBcXExZWVkEh+5dfX191I41WKRimyE1252KbYbUbHc82ryqJpyUlq9die2OfNbFnp1tOAdPPldGbsbRDVVKxfdaRFLPq+U1mMHJk4bHOxTpp9zMdD6ycBKPr6zkOw+vZsGUokOWcmntZZ3SaMhKT6MpSXpKIxmjtRC41MzeDWQDhWZ2N1BlZmOcc5VmNgao9vavAMZ1en4psOvwgzrn7gDuAJg/f75btGhR/1vRSVlZGdE61mCRim2G1Gx3KrYZUrPd8WhzwzuVsGQZZ5x2EseMjvzX++olO/jHunc4bv4plA6LbG217qTiey0iqaU1EOKFddXMKRnC0NzMeIcjR8HnM/73qhM446cv8MiKXdx09tSOx1qDbkB6Svc3db8sTXea24Is3rKXyUV5FOZkMCSnf4WWoqnXM+Wcu9k5V+qcm0i4gNHzzrlrgEeA67zdrgMe9m4/AlxlZllmNgmYBrwZ9chFRCSq2ueU9rn6rre/KvCKiPTuh4+vYUXFAS4/QbPbksG44bnMmzCMR1cc2gfXFgyRNQBzSvuzJMwtj67muj+/yRk/fYFzf1HGlpqGGETXN0dzpn4MnG9mG4Hzvfs451YD9wFrgCeBm5xzydGvLCKSxPzN/ay+mxPeXxV4RUR69+KGPZw3s5iPnj4p3qFIlFwwu5h1u/1UHmjq2BaeUxr/6ru7DzTz1X+t4LsPr2JDlZ+fPLmOfy2pACDdZ9TUt3L3G9sOec6+hlaeXVPF3xdvpyU4MPUi+vTNwzlXBpR5t2uBc7vZ71bg1qOMTUREBpDfWxImP7PvS8IAqsArItKL6rpmttU2cvUp4+MdikTR6VNHAut4ZWMNV84Pz2IcmDmlXa9T+tTq3fz+xU3c+ZGT+cFja3h8ZSVmcOfr2zCD9xw7hu9fNochORl84m9Lue+tHZx7zChOmTyCjdV+3n/7azR4FYUnD/Fx1plBsjNiu8Jn3755iIhI0vI3t5GflY7P17dfdtvnorRX7xURkSPta2jl5B89B8C8CcPiHI1E0zGjCyjKz+LxlZVcOX8coZAjEHKxXxKmm+q7v3x2I2sr67jid6+zvsrPRxZO4kOnjOOJlbu5aM5ophUXdOz7sTMm8dqmGj70x8XMLimkYl8T2Rlp/PzKubQEQnzh3uXc8dJmPnfutJi2JbZnSkREBg1/c6DPQ3dBPaUiIpFYU1kHwOySQuaWDo1vMBJVPp/xsTMmUbZ+D79/cRNtoXCiGPOe0oy0I+aUtgVD7NjbCMD+plZGFWRxzanjmTqqgM+dO+2QhBTg1MkjeOOb53La5BGs3lVHZrqPez9xGhcdO4bLTxjLF07M4hNnTY5pO0A9pSIi4vE3t/UrKc3P1pxSEZHebNpTD8Cfrz+J9BgnKzLwPnr6JJZt28d//2ddR0945gAN33XOYRYe5bRix37qWwL87uoTuejYMREdpzA7g+9dOpur//gGv/ngCUwdld/x2PGj0slKj+3QXVBPqYiIeOpbAuRn9T0pTfMZBVnpHFBPqYhIt8qr6ynISmdUQVa8Q5EYyEjz8d1LZwPw0sYab1usCx35CDkIhA4WI3p5Y3gN3NOmjOjTsWaMLmDJt8/nlMl9e160KCkVERGgffhu/9YqK8zJ0JIwIiI92LSnnsmj8jt6tCT5jB2aQ2F2Ov/73EYAMmPcw9jeg9l5Xukr5TUcN3bwrYGrpFRERID+zymF8DIyGr4rItK19bv9vLV1H8eOLYx3KBJjF805OGS2sTW2P9ZmZYRTuf+srOStrXvZfaCZt7fv46zpI2P6urGgOaUiIgJEo6dUSamIyOHW7Krjfb97lfys9JhXMJX4+84lszhn5ig+8belzBoT2x8hsrzqvl+9/x0A8jLTCDl4/7zSmL5uLKinVEREgP4XOoJwkQQtCSMicqQnVlbSFnQ8fNNCRhVkxzscibG8rHQumD2a8lsvYsHUopi+1hnTRnYkpnNLh9DQGuQ9x41hwoi8mL5uLKinVEREaA2EaAmEKOhHoSOAwpx01laqpzTRmdlQ4I/AHMABH3HOvR7XoESS3OIttcwZO4Rxw3PjHYoMoIGosFwyNIfXbz6XhpYAw/MyeXv7fhb0scBRolBPqYiI4Pfmgx5dT6mS0kHgV8CTzrljgLnA2jjHI5LUlm7bx4odBzh10vB4hyJJanheJuOG55KXlc7p04rw+QZnIS31lIqICPUt4aG3+Ucxp7S+JUAo5AbtBTHZmVkhcCZwPYBzrhVojWdMIsmq8kATX75vBYu37KVkaDZXnzIh3iGJJDT1lIqICH5vPmj/e0rTcQ78LZpXmsAmA3uAv5jZ22b2RzMbfBOPRAaBf7y5g9c21XL58WN57LNnMH6Ehu6K9EQ9pSIi0jH0tt9JaU64h7WuqY0hOf3rbZWYSwdOBD7rnFtsZr8CvgH8V/sOZnYjcCNAcXExZWVlUXnh+vr6qB0rWekcRWYwnKeQc/zz9SZmDvdxyah9vL341QGPYTCcp3jTOYrMQJ0nJaUiIkK911Na2N/hu97zNK80oVUAFc65xd79+wknpR2cc3cAdwDMnz/fLVq0KCovXFZWRrSOlax0jiKT6OfphfXVfOQvbwFwy3tPYNGxY3p5Rmwk+nlKBDpHkRmo86ThuyIi0jF8N/8oqu8C1DVp+G6ics7tBnaY2Qxv07nAmjiGJJJ0nl5dBcC1p07gwjmj4xyNyOChnlIREYlK9V1QT+kg8FngHjPLBDYDN8Q5HpGkUl7t5+SJw/nB5XPiHYrIoKKkVEREDvaU9jMpHdJpTqkkLufccmB+vOMQSUYV+xpZufMA7zuxNN6hiAw6SkpFRIT6lgCZ6T6y0tP69fyDPaUavisiqWdfQyun/+QFACYXqai1SF9pTqmIiFDXHKCwn72kcLCHVT2lIpKK7l2yo+P2/InD4xiJyOCknlIREcHf3EZBPyvvAqT5jIKsdM0pFZGU09wW5E+vbGHBlBH8+fqTyM7o34gTkVSmnlIREaG+JdDvIkftCnMyVH1XRFLOI8t3scffwufOnaaEVKSflJSKiAj+5kC/l4NpV5CtnlIRST1v79jPkJwMTpmkYbsi/aWkVEREvOG70egpVVIqIsmtuS3IzQ+u5O3t+wBYt7uOmWMKMLM4RyYyeGlOqYiI4G8OHNWcUghX4N25vylKEYmIJKbn1lbzjze38483t3P1KeN5e/t+rl8wMd5hiQxq6ikVERHqozB8tzAnXT2lIpL0HntnFwDzJgzjnsXbAZhVUhjPkEQGvV6/gZhZNvASkOXtf79z7rtmNhy4F5gIbAX+n3Nun/ecm4GPAkHgc865p2ISvYiIHLVQyFHfenRLwkC4p1RzSkUkmW2taeDpNVV87PRJfPviWTS3BXl6TRUXzh4d79BEBrVIekpbgHOcc3OB44ELzexU4BvAc865acBz3n3MbBZwFTAbuBC43cxUikxEJEHVtwZwjqMfvpuTQX1LgFDIRSkyEZHE8qvnNpKRZtx41mQAsjPSuHRuCZnpGnwocjR6/T/IhdV7dzO8/xxwGXCnt/1O4HLv9mXAP51zLc65LUA5cHI0gxYRkejxN4eXcTnqQkfZ6TgH/hYtCyMiyefBZRU8vHwn1502kVEF2fEORySpRPSzjpmlmdlyoBp4xjm3GCh2zlUCeH9HebuPBXZ0enqFt01ERBJQvZeU5keh+i6geaUiklScczy8fCdfum8Fs0oK+eRZU+IdkkjSiegbiHMuCBxvZkOBh8xsTg+7d1UP+4ixXGZ2I3AjQHFxMWVlZZGE0qv6+vqoHWuwSMU2Q2q2OxXbDKnZ7oFs88Z9QQC2rF9D2d4N/T7Ojqpwcvv8K68zobB/szZS8b0WkcTUEgiSZsbvX9rMz55az4ziAh781EIN1RWJgT79LO6c229mZYTnilaZ2RjnXKWZjSHciwrhntFxnZ5WCuzq4lh3AHcAzJ8/3y1atKjv0XehrKyMaB1rsEjFNkNqtjsV2wyp2e6BbLNbVw2L32LhKfM4cfywfh8na1Mtv377DabPPp7Tpozo1zFS8b0WkcT0pXtX8PjKSgBOnjic3187TwmpSIz0+n+WmY30ekgxsxzgPGAd8AhwnbfbdcDD3u1HgKvMLMvMJgHTgDejHLeIiERJ+xzQgigsCQOoAq+IDHrOuY6EdHheJj987xyG5WXGOSqR5BXJN5AxwJ1eBV0fcJ9z7jEzex24z8w+CmwHrgRwzq02s/uANUAAuMkb/isiIgnI7yWRR119N1tzSkUkOeyuawbg+5fN5sOnTYxvMCIpoNek1Dn3DnBCF9trgXO7ec6twK1HHZ2IiMRc1Krvthc6alb1XREZ3NbsqgNg1pjCOEcikho0MF5EJMX5m9vwGeRmHt2S0gVZ6Zipp1REBr+3tu7DZzBjdEG8QxFJCUpKRURibM2uOnbsbYx3GN2qbw6Qn5WOWVfF0yPn8xn5WemaUyoig1pbMMQDyyo455hRRz2tQUQio6RURCTGPnn3Um55dHW8w+iWvzkQtS9ehdkZ1DVp+K6IDF4PLqtgj7+Fq0+ZEO9QRFLG0U0gEhGRHtW3BNi+t5GWQOLWe6trDhz1fNJ2hTkZ6ikVkUGrsTXAbc9s4MTxQ1k0Y2S8wxFJGeopFRGJoU3V9QBU1bVQU98S52i6Vt/S1lE592gVZqdrTqmIDFp/eXUrVXUt3PzumUc9pUFEIqekVEQkhjZU+Ttut1dzTDT+5gD5Ue0p1fBdERl8lm7bx23PbODcY0Zx0sTh8Q5HJKUoKRURiaHy6nrSfeFf29dUJm5SGrXhu9kZ6ikVkUFn3e463v+71wiGHJ8+e0q8wxFJOZpTKiISQxuq/EwdlY+/OcDqhO0pbYvinFJV3xWRwefF9XsA+Mn7j2XeBPWSigw0JaUiIjG0sbqeE8cPo6ktyJpdB+IdzhGcc9S3BMjPil713fqWAKGQw+fTfCwRGRxeKa9h2qh8PnDS+HiHIpKSNHxXRCRGGlsDVOxrYtqofGaXFLK5poHG1sSab9kSCNEWdFGtvusc+FsSq50iIp0559ixt5E/vbKFy3/7Ki9vrOGcmaPiHZZIylJPqYhIjJR7lXenFefjM8M5WFvpZ96EYXGO7KD2obaFUZtTGj5OXVMbQ3K06LyIJJ4HllZw/9IKXt9cC4AZDM/L5FNnaS6pSLwoKRURiZGNVe1JaQHZGWlAuNhRIiWl9V6l3IJoLQnjJaIHmtoYF5UjiohEz/baRr78rxUAfOn86bz72DGMLMiipS3I0NzMOEcnkrqUlIqIxMiGaj+ZaT4mDM8lzWcMyclIuHmlfi8pzc+KXvVdQMWORCQhvb65BoCnvnAmM0YXHHxAIztE4kpJqYhIjJRX1TN5ZB7paeHp+7NLChNurVJ/R09p9KrvAtQ1aU6piCSe1zfVMrIgi+nF+fEORUQ6UaEjEZEY2Vhdz9RRB7/4zBpTyLrdfgLBUByjOpTf69GM2vBd9ZSKSIKqbwnw7Npqzpw2EjNVBxdJJEpKRURioKk1yI59jUwbdXB42OyxhbQEQmyuaYhjZIdqr5Ibzeq7EC50JCKSKA40tfHJvy2lviXAtadNiHc4InIYJaUiIjGwaU89znHIELFZY4YAsDqB5pVGe/huQVY6ZlDXrOG7IpI4vv/oGl4pr+EzZ0/l+HFD4x2OiBxGSamISAxsqPID4eVg2k0ZmUdmui+h5pW2D9+NVqEjn8/Iz0pXT6mIJIzlO/bzwLIKPr1oCl+5YEa8wxGRLqjQkYhIDGysricjzZgwIq9jW3qaj2NGF7A6oZLSALmZaR3FmKKhMDtDc0pFJCE8vXo3tzy6hhF5mXz67KnxDkdEuqGeUhGRGNhYVc+kojwyDkv2ZpcUsqayDudcnCI7VH1zIGq9pO0KczJUfVdE4m5jlZ8b/7aU/Kx07vjw/Kh/1olI9CgpFRGJgY3V/kOKHLWbNaaQ/Y1t7DrQHIeojuRvaYvafNJ2hdnp6ilNUGaWZmZvm9lj8Y5FJNY2VtcD8MurjmfehGFxjkZEeqKkVEQkyprbgmzf23jIfNJ2s0rCxY6iOa90b0MrX7x3OTX1LX1+rr85ELXlYNqFe0qVlCaozwNr4x2ESKw8t7aKP72yhYaWABX7GgEYOywnzlGJSG+UlIqIRFl5dbjyblc9pceMLsAsuhV4H19ZyUNv7+TBZRV9fm44KY12T2lGR1VfSRxmVgq8B/hjvGMRiYa9Da186b7l3Pr4GoIhR1swxEfvXMIPHlvDv5fvZOe+Jgqz0zvWTxaRxKXB9SIiUVbuDRnrqqc0LyudSUV5Ue0pfWXjHgD+s2o3N545pU/P9Te3UTI0O2qxABTmqPpugvol8DXgyF9LRAahO17azIPLdgLwh5e3MCTnYPK5Ysd+autbGTssN17hiUgfKCkVEYmyjdV+0n3GxE6VdzubNaaQt7fvj8prBYIhXiuvJSvdx9vb97P7QDOjh0SeZPqbAxRkRXn4bnYG/pYAwZAjzWdRPbb0j5ldDFQ755aa2aIe9rsRuBGguLiYsrKyqLx+fX191I6VrHSOItN+nloCjntea+TEUWlMHuKjoj7EG5XhH8PGFfh4ff0unHOMzPWl5HnVv6fe6RxFZqDOU69JqZmNA+4CRgMh4A7n3K/MbDhwLzAR2Ar8P+fcPu85NwMfBYLA55xzT8UkehGRBLShqp6JReE1Sbsyu2QIj71TyYHGNobkHl1CuKLiAP6WAF88bzr/8+wGnl6zmw+fNjHi58dk+K7XW1HfHDjq9knULAQuNbN3A9lAoZnd7Zy7pvNOzrk7gDsA5s+f7xYtWhSVFy8rKyNax0pWOkeRaT9PP31yHf62TXzr/Sczb8JwAD77j7fZWtPAohkj+fXz5QBcPG8iixYdE8+Q40L/nnqncxSZgTpPkcwpDQBfds7NBE4FbjKzWcA3gOecc9OA57z7eI9dBcwGLgRuN7O0WAQvIpKIyqvrmd7F0N12s0oKAVhdefTzSl/euAcz+PBpE5gyMo//rNwd8XMDwRBNbUHyY1B9F1AF3gTinLvZOVfqnJtI+Br9/OEJqchgsMMf4orfvcbtZZt4/4mlHQkpwP9edTyPfGYh7zuxlLNnjOSKeaV8+uy+TWkQkfjo9ZuIc64SqPRu+81sLTAWuAxY5O12J1AGfN3b/k/nXAuwxczKgZOB16MdvIhIomluC7KttoFL5pZ0u8+sMeGkdM2uOhZMKTqq13t5Yw3HjR3CsLxMLpozhtvLytnb0MrwvMxen1vfEi5GFIvquwAHmtoYF9Uji0gqcs7xjzd3cN+SHaysaKIwJ8AnzpzMl98145D9zMLTBSYV5fGXG06OR6gi0k99qr5rZhOBE4DFQLGXsLYnrqO83cYCOzo9rcLbJiKS9DbvaSDkYNqo7ntKRxZkMaog66iLHdU1t7F8x37OmDYSgAvnjCbk4Nk1VRE9v71Cbiyq77bHJ4nHOVfmnLs43nGIRGJfQyu/eHoD33xoJYFQiAsmZvDE58/g5nfP7HaKhIgMPhF/EzGzfOAB4AvOubr2X6O62rWLba6L46mYQpSkYpshNdudim2GwdXuN3aFE70D29dRtm9Dt/uNzg7w5sZdlJXt7/LxSNq8tCpcTCi/oYKyskqccxTlGHe/tJpRDZt6jXV7XRCAbeXrKavvff9IbfOO+/qS5bTu6FvCO5jeaxGJvc/8Yxmvltdy7NghPHzTQl566UXGDNG6oyLJJqJvC2aWQTghvcc596C3ucrMxjjnKs1sDFDtba+AQ0ZslQK7Dj+miilETyq2GVKz3anYZhhc7V7y1HrSfJu48qKzyErvfjr9kpb1/O7FTZy68AyyM47cL5I2P/vvleRm7uQjl57d0WNwecMa7np9GyeeurDXtfkWb66F197gtHnHc/q0oxtG3FnFvka++9oLjJsyg0Xz+zaAdzC91yISO8GQ47cvlPNqeS3HjC7gtx86EZ+qeYskrV7HPVi4S/RPwFrn3G2dHnoEuM67fR3wcKftV5lZlplNAqYBb0YvZBGRxLWx2s+EEbk9JqQQLnYUDDk2VtX3+7Ve2VjDaZNHHDKE7cI5o2kNhnhhXXUPzwyL2fBdb06p1ioVkb5yzrFzfxO/eHo9tz2zgfysdO7+2CmMH6H1RkWSWSTfRBYC1wIrzWy5t+2bwI+B+8zso8B24EoA59xqM7sPWEO4cu9NzrlgtAMXEUlEG6vqmV5c0Ot+s9sr8O46wLGlQ/r8Ojv2NrK1tpHrFkw8ZPuJ44cxsiCLJ1ft5rLje57O728JJ43RTkrzM9Mxgzov6RURidS/l+/ki/euAODsGSP58/Un0cOUMRFJEpFU332FrueJApzbzXNuBW49irhERAadlkCQrbUNvOe4Mb3uO25YLvlZ6azuZ7GjlzfWAHQUOWrn8xkXzC7mgaU7aWoNkpPZfY9tvZc0RntJGJ/PKMhKV0+piPRJc1uQ3zxfzoi8TG77wPGcNnmEElKRFKGyZSIiUbKlxqu8G0FPqc9nzBpTyJrK/ialexgzJJspI/OOeOzC2WNoagvy4oY9PR6jvSezt7mn/VGYk6HquyISkea2IA8uq+CSX7/Cpj0N/Pf7juWs6SNVXVckhej/dhGRKNngzQ/taTmYzmaVFLK2so5g6IgC5T0KhhyvltdwxrSiLnsRTpk8nKG5GTy1enePx/E3B8hIM7Ji8MWvMDuDuiYN3xWRntU1t3HOz8v40n0rSPMZ/3fNibxr9uh4hyUiAyy6Y7ZERFJYeZUfn4UXbo/ErJJCGluDbKttYPLIyBJZgHcq9lPXHDhi6G67jDQf580s5qnVu2kNhLrtbfA3t5GflR6T4XGFOenqKRWRXt3zxnZ2HWjme5fM4trTJpKmCrsiKUk9pSIiUbKhqp6JI/K6XOKlK7PGtBc76tsQ3pc31mAGC6d2v4zLRXNG428O8Nqmmm73qW8JUBCDobvQ3lOqpFREuuac4xdPr+cnT65j3oRhXL9wkhJSkRSmnlIRkSjZWO1naoRDdwGmFxeQkWasqazjkrklET/vlY01zCkZwvC8zG73WTi1iLzMNJ5avZtFM0Z1uY+/ORD1yrvtCnMyOpacERFpt253Hf98cwf/Xr6T/Y1tXDq3hFsunR3vsEQkzpSUiohEQWsgxNbaRi6a03vl3XaZ6T6mjiroU0+pv7mNZdv3ceOZk3vcLzsjjbOPGcXTq6v44eWuyx4If3Nb7JJS9ZSKyGHuWbyNbz20isw0H+fPLubMaUW894RSFTQSESWlIiLRsKWmgWDIMa048p5SCK9XWra+5yq5nb2xeS+BkOP0ad0P3W130ZwxPPZOJW9t3cupk0cc8bi/OUDpsNgsSF+Yk46/JUAw1HVCLCKpIxRy/PnVLfzw8bWcc8wofnHlXIb1MNJDRFKPfpoSEYmCjdV+gD4N34XwvNKa+haq65oj2v+VjXvIyUhj3oRhve67aMZIstJ9PLmq6yq8/uYAhTHsKYWDa6GKSOq68/Wt/PDxtbxrVjG/u+ZEJaQicgQlpSIiUbChqh6fwZQ+VNGFcE8pwOoI1yt9eWMNp0weTlZ678WU8rLSOXP6SJ5avZtQF8vOxHT4bk44Kd11oCkmxxeRwaG5LcjtZZs4bfIIfn/tvIg+u0Qk9SgpFZGE5ZzjL69uoaohFO9QelVe7Wf88NyIK++2m+klpWsimFdasa+RzTUN3S4F05ULZ4+m8kAz7+w8cMh251xMq++eMa2I3Mw0bntmQ0yOLyKDw39WVbLH38JNZ0+NyfJTIpIclJSKSMJaU1nHLY+u4aHy1niH0quNVfVMHVXQ5+cVZmcwfnhuREnpKxvDy7ucGcF80nbnzSwm3WdHDOFtbA0ScpAfo57S4sJsPn/uNJ5ZU8Xz66pi8hoikthCIcedr21jwohcFkw5cl67iEg7JaUikrAeXVEJwNKqIAcSuJJrayDElpoGpvexyFG7WWMKWb3rQK/7vbyxhuLCrD7NWx2Sm8FpU0bw5KpKnDs4hLd9uZZYDd8FuGHhJKaOyud7j6yhuS0Ys9cRkcT02xfKWb5jPzedPRWfCp6JSA+UlIpIQnLO8dg7uxg3PIe2EDz+TmW8Q+rWttoGAv2ovNtudkkhW2sbqW/pvihQMOR4dVMNZ0wb2echcBfOGc3W2kbWV/k7tvmbw0l+rIbvQnjJm+9fOpvtexv5/YubY/Y6IpIYgiHHOxX7qW8J8NqmGv7n2Q1cfnwJV84rjXdoIpLglJSKSEJavmM/Ffua+Nw50yjJNx5YVhHvkLq1oaoegGn9GL4LMMubV7q2h2JHq3YeYH9jG2f0Yehuu3fNGo0Z/GflwSG8fi8BLsiK7cpgC6YWcfFxY7i9rJzttY0xfS0RiY9d+5u47ZkNXPLrV7j0N68y7wfP8NG/LmFiUR63vvdYzSUVkV4pKRWRhPToikoy03xcMGc0p49NZ+m2fWzeUx/vsLq0sdqP9aPybrvZJUOAnosdvbwxvJbpwql9T0pHFmRx0oThPLW6U1I6AMN32337PbNI8xnff2x1zF9LRAbWY+/s4oJfvsRvnt9Ims/42oUzOGNaEfMmDOOOa+eTF+MfvkQkOeiTQkQSTijkeHzlLs6aMZLC7AwWjEnn/g1tPLCsgq9ecEy8wzvCxup6xg3LJSezf0sdFBdmMTwvs8d5pS9vrGF2SSFF+Vn9eo0L5ozmB4+tYUtNA5OK8gZk+G670UOy+cJ50/jRE+t4dk0V580qjvlrikjsba9t5PP/XM6xY4fwv1edwPgRufEOSUQGKfWUikjCeWvrXqrqWrhkbgkAQ7N9nDl9JA8u20mwi/U2421jlb/fRY4AzIzZJYWs6Wb4bn1LgGXb93F6P4butrtwzmiAjiq8A9lTCuGiR9NG5XPLY6tV9EgkCTjn+MUz60kz4/fXzlNCKiJHRUmpiCScR9/ZRXaGj3OPGdWx7Yp5pVQeaOb1TbVxjOxIbcFw5d3+LAfT2aySQjbsrqcteOSarIs319IWdJzZh/VJDzd2aA7HlQ7hSW8Ib72XlMZqSZjDZaT5uOWy2ezY28T/vbhpQF5TRGLn9rJNPLx8F59aNIXiwux4hyMig5ySUhFJKIFgiP+s3M25M4sPmYt03sxiCrPTuX/pjjhGd6RttQ20BR3T+rBMS1dmjSmkNRiivPrIebMvb6whO8PHvAnDjuo1LpwzmhU79rNrfxP+5jbMID9z4GZxLJhSxKVzS7i9bJOKHokMYv9+eyc/e2o9lx9fwhfOmxbvcEQkCSgpFZGE8vrmWmobWrnkuDGHbM/OSOOSuSU8uXp3x3zIRLDRq7w7vfjoekrbix2t7qLY0csb93DypBFkZ/Rvzmq7C2eHh/A+tXo3dc0B8jPTB3ztwG+9ZyYZPuOWR1X0SGQwWr/bz9ceeIdTJw/np1fMVWVdEYkKJaUiklAeW1FJflY6i2aMOuKxK+aV0twW4omVibNm6UavZ3PKqLyjOs6kojxyMtKOqMC7a38Tm/Y0cOZRzCdtN3lkPtOL83ly1W7qWwIDNnS3s+LCbL54/nSeW1fNs2uqBvz1RaR/QiHH3xdv54rfvUZhdga/+dCJZKbra6SIRIc+TUQkYbQGQvxnVSXvmlXcZa/g8eOGMmVkHvcvTZw1SzdU+Rk3PIfcoxwGm+YzjhlTcEQF3lc21gBwxlHMJ+3swjljeGvrXrbWNAxYkaPDXbdgItOL8/neoyp6JDIYOOf4xN1L+eZDK5kzdggPfOq0flcCFxHpipJSEUkYL2/cQ11zgIvnjunycTPj/fNKeWvrPrbWNAxwdF0rr65n2lEWOWo3a0y4Aq9zBysMv7RxD6MKso6qum9nF84eTcjBkm37BmQ5mK5kpPm45dI5VOxr4vYyFT0SSVTOOZ5ZU8Xkbz7BM2uq+OoFM/j7x09hwoijGxkiInI4JaUikjAee6eSITkZnD61+17B951Qis/gwWXx7y0NBENs3tNw1EWO2s0uGYK/OUDFviYgPFzu1fIaTp9WFLV5WzPHFDDBW7ohXj2lAKdNGcFlx5fwfy9uSpgfGETkoMbWAB/4/Rt8/K4lOAfvO2EsnzpriuaQikhMKCkVkYTQ3Bbk6dW7uXD26B7nKY0eks3p00bywLKdhOK8Zum2vY20BkNMO8oiR+1mlRQCdAzhXb2rjn2NbZwRhfmk7cyso+BRflb8klKAb757JplpPm55dPUhvcMiEn8/eGwtb23byyfOnMyy/zqf2z5w/IAXRhOR1NFrUmpmfzazajNb1WnbcDN7xsw2en+HdXrsZjMrN7P1ZnZBrAIXkeRStr6ahtYgl8wt6XXf9584lp37m3hjc3zXLG2vvButntJjRhfgMzqKHb20cQ8AC6dGLykFuGBOOCmN1/DddsWF2XzhvGm8sH4Pz66tjmssInLQG5tr+edb2/nY6ZO4+d0zGZ6XGe+QRCTJRdJT+lfgwsO2fQN4zjk3DXjOu4+ZzQKuAmZ7z7ndzI5uDQMRSQmPrqhkRF4mp04e3uu+F8weTUFWOvfHeQjvxio/AFOjlJRmZ6QxZWR+x7Iwr2ysYeaYQkYVRHdh+uNLh3La5BGcOH5oVI/bH9ctmMiM4gK+98hqmlpV9Egk3qrrmvnsP95m0og8vnDe9HiHIyIpotek1Dn3ErD3sM2XAXd6t+8ELu+0/Z/OuRbn3BagHDg5OqGKSLJqaAnw3Loq3n3sGNLTev+tLDsjjYvnlvCfleGlTeJlY3U9Y4fmkBfFYbCzS8LFjloCjiXb9kZ16G47n8/4x42ncuX8cVE/dl9lpPn4/mWz2bm/id+Vlcc7HJGU5pzjK/e/Q31zgN9dMy+qn20iIj3p76dNsXOuEsA5V2lm7QsKjgXe6LRfhbftCGZ2I3AjQHFxMWVlZf0M5VD19fVRO9ZgkYpthtRsd7K2+Y1dAZrbQpSGqigrqzni8a7aPdmCNLUF+Z9/vcAZpfEZhvr25iZGZFlU35OspjYqD7Ty6nZHW9AobNxFWVnyr+d5Wkkat79QzrdOdJCE/8ZFEt2aXXV85h/L2Lynge9cPIsZo6MzV15EJBLR/gmsqxnwXVavcM7dAdwBMH/+fLdo0aKoBFBWVka0jjVYpGKbITXbnaxtvvvOJYwuPMDHLz+ny0IaXbX7LOf4+6YXWdWQxX8tOm2AIj0oGHJUPfskF50wgUWLZkbtuBmlNdy7fjHP7/KRlW587LJFXa7ZmmxmndjMub94kbX1xg1XLop3ODJAQiGHI7xO74GmNpZt28eqnQe44fRJcS/ElUrqWwJc95c3OdDUxqVzS7j2tAnxDklEUkx/P/GrzGyM10s6BmivUFEBdB4PVgrsOpoARSS5HWhq46UNe7j2tAl9quxoZlwxr5SfPbWe7bWNjPeWORko2/c20hoIRW0+abtZY8IVeCvqHWdMG5ESCSnAqMJsnv3yWaxd9kbvO8ug5JzjsXcq2bGvkabWIFtqGnhmTRXpPuPcmcW8sbmWan8LADmZaXzsjMlxjjg1rK2s46dPrmOPv4WHPr2AE8YP6/1JIiJR1t+k9BHgOuDH3t+HO23/u5ndBpQA04A3jzZIEUleT6/eTWswxMXHjenzc997wlh+/vR6HlhWwRfPH9iCHBu8IkfRqrzbblheJiVDstl1oDkm80kTWXFhNmvjHYRE3U+eXEdDdRt337Wko8qyGWSl+7hg9miyM3w8uqKSrAwfv7hyLr9+fiO/K9vErv3NHFtayGVzx2opkhhYtfMAP3lyHS9vrCEzzcfHz5ikhFRE4qbXpNTM/gEsAorMrAL4LuFk9D4z+yiwHbgSwDm32szuA9YAAeAm55zKKYpItx59p5LSYTkcP25on59bMjSHhVOKePDtCj5/7rQB/eJaXu0tBxOlNUo7m1VSyK4DzZw+dWTUjy2py8zGAXcBo4EQcIdz7lexfM265jb++eZ29jW2UZC9l69feAzXnDqeUAgKc9IxC/8/+9/vO46Qc2Sk+TCDL923gj+/ugWAu17fxhfPm87pU4uUnEbJ0m37+PhdS/AZfPWCGVx9yniG5mrZFxGJn16TUufcB7t56Nxu9r8VuPVoghKR1LC3oZVXy2u48czJHV9O++qKeaV84d7lvLl1L6dOHhHlCLu3scpPyZDsmMx7e9fs0WzfXcMxKjQi0RUAvuycW2ZmBcBSM3vGObcmVi9YmJ3Bkm+fz7/+8wLvveAsstK7Ho6e5jPSvLIU7zuxlAVTishIM55fV81Pn1rPh//8JqdMGs7vr52n5Okoraw4wJX/9xoj8rO498ZTmTwyuqM9RET6I5J1SkVEYuI/qyoJhly/hu62u2D2aPKz0rl/6cCuWbqhqj4mvaQA/2/+OL55So56hSSqnHOVzrll3m0/sJZuKuRHU5rPGJ3n6zYh7croIdmMyM/iyvnjePlrZ/Oj9x7L29v388E/LObO17ayY29jDCNOXjv3N3HzQ+8wJCeDZ794lhJSEUkYSkpFJG4eW1HJ5JF5HcV9+iMnM433HDuGJ1ZW0jBAa5YGQ45Ne+qjPp9UZKCY2UTgBGBxnEPpVXZGGh86ZTz/d+2J7PG38N1HVnPWz17gk39bymubagbs//vBrC0Y4h9vbuf9t7/Glj0N/Oi9xzIkNz5LaYmIdEX11kUkLqrrmnljSy2fPWdav4futrtifin3LtnBk6t28/55pVGKsHs79jbSEggxrVhJqQw+ZpYPPAB8wTlXd9hjCbuGuA/48YI0aptyeKkiQNmG3Ty5ejc+g7H5PkoLjKuPySI/c3COMIjVOtT/Lm/l3+VtAAzNMr48L4uc2vWUla2P+msNhGRdrzvadJ56p3MUmYE6T0pKRSQuHl9ZiXNwyVEM3W03f8IwJozI5f6lFQOSlG6MYZEjkVgyswzCCek9zrkHD398sKwh/gGgqTXI8+uqWbXrAGsr6yhbv4dpE0by/XfNidrrDKRonyPnHA8s28m/y1dw8sThfHLRZM45pjhqx4+XZF2vO9p0nnqncxSZgTpPSkpFpN9q61uoqW9lRj8K8jz2TiXHjC6ISmJnZrz/xFJue2YDFfsaKR0W2zVLN1aHl4OJ9hqlIrFk4SEJfwLWOudui3c8RysnM433HDeG93g/bH373yu5Z/F2Lpg9moVTU2s5pc4CwRD3LtnBvW/t4J2KA8yfMIy7P3YKmemasSUiiUufUCLSL/UtAT5wxxtc8MuX+OTflrJpT33Ez925v4ml2/ZxydySqMXzvhPD9VoeXLYzasfszsaqesYMyaYwW3OyZFBZCFwLnGNmy73/3h3voKLl6xcew9SR+Xzq7qXc9fpW7nxtK0+t3s2BxrZ4hzagHl9ZybceWsWaXXV8690z+dP1JykhFZGEp55SEekz5xxfuW8Fm/fUc/Up4/n32zt5Zm0VV500js+fN41RBdk9Pv/xd3YBHFXV3cOVDstlwZQR3L+0gs+eM/Wo56n2ZGO1X72kMug4514BBueEywgUZGfwx+vmc9Udb/Cdh1d3bE/zGWVfWcS44bEdQZEo/vHmdsYMyabsq4v6VPFYRCSe9NOZiPTZ7WWbeHL1bm6+aCa3vvdYyr56NlefMp5739rBop+VcdszG6jvoSLmoysqOa50CBNG5EU1rivmlbJ9byNvbd0X1eN21twWpLy6nmmjNJ9UJNGMG57Ly187m8XfPJeXvno2Xzp/OsGQ443NtfEObUAs3baPNzbv5foFE5WQisigop5SEemTFzfs4edPr+eSuSV87IxJAIwsyOL7l83hhoWT+PlT6/nf5zZyzxvb+Px50/jgyePJSDv4+9fWmgZW7jzAt949M+qxXThnNP/171V85+FVnDxpOMPzMhmRn8WIvMzwbe/v0NxM0rpYA9Q5R11TgJ37m8L/7Wtk14Fmdu5r6ti2x98CwDFjlJSKJCKfzyguDI/W+MzZU/nDS5tZvmM/V84fF+fIYqe5Lci/luzgx/9Zx/C8TK45dUK8QxIR6RMlpSISse21jXzuH28zo7iAn7z/2COGyE4qyuO3V5/Ix3fs57+fWMt3Hl7Nn1/ZwlcumMF7jh2DmfGYN3T3PVEcutsuNzOdz547jfve2sG/395JXXPXvbU+g2G54QR1eF4m2Rlp7D7QzM79TUf08Gal+xg7NIeSoTmcM2MUJUNzGD8ih4vmRD9+EYkun884btwQ/rWkgusWTGR6ElbMDoYcl/3mVdZX+Tlt8gi+ffFM8rL09U5EBhd9aolIRBpbA9z4tyU45/j9tfPIzez+4+P4cUP5542nUrZ+Dz/+zzo+8/e3+UPpZr5x0UweXVHJ/AnDKBmaE5M4P3nWFD551hQgvGD8voZWahta2dv+t76F2o7b4e37GluZMCKX06aMoHRYOAEdOzSHscNyGJGXGdP5qSISWzcsmMSr5bV89V8rePgzp8c7nKh7YFkF66v8fPG86XzmnKldjgIREUl0SkpFpFfOOb7xwErWV/n5y/UnRTQX1Mw4+5hRnDl9JA8uq+C2ZzbwwT+8AcAtl86OdcgAZKT5GFWYzajCngsviUjyOm9WMd+9ZBa3PLqGsvXVFGSn89bWfbS0hZgxOp8LZo8elD88Oef4r4dXcfcb2ykdlsOnz56ihFREBi0lpSJxsmt/E3sbWpk6Kp/sjMQuSPGnV7bwyIpdfPWCGSyaMapPz03zGVfOH8clc0u487WtlK3fE9WlYEREevO+E0r5w0ubuf4vbx3x2Anjh/LbD50Ys9EbsbKhqp6739jOVSeN40vvmn7I3H0RkcFGSalIHDy6YhdfvX8FzW0h0nzG5KI8Zo4p9P4rYNaYQkYWZCXEr/evbarhv/+zjgtmF/PpRVP6fZzsjDQ+cdYUPnFW/48hItIfQ3IzePSzp/PEqt0U5WVy2pQR5Gel89DbO/n+o2t4/+9e42NnTOaiOaMZXZiNbxD0OK7ceQCAj50xuddluEREEp2SUpEBFAo5bntmA795oZx5E4Zx3YKJbNjtZ21lHUu27uWRFbs69h2Rl9mRpLYnrC1BR1NrkKBzhJzDhei4HXIO58JFL9pvp/mMMUOy+53c7tzfxGf+/jYTR+Ty8yvnJkSSLCLSHyPys7j2sKq0V84fx+SRedz6+Fp+8NgafvDYGjLTfNywcCI3x6BCeDSt2nmAnIw0JhVFd2ktEZF4UFIqMkD8zW188d4VPLu2ig/MH8f3L58dXkdu7sF99je2srYynKSuraxj7e467nx9G62B0MGdnnmyT687t3QI1y+cyHuOLSEzPfLhXc1tQT75t6W0BkLc8eH5FGRn9Ol1RUQGg3kThvPgpxfy9vZ9rKms4+UNNfz+pc28tqmWi44dzbzxwyjIzmDX/ibOmjEyIYbJlq2v5q+vbWXehGGaRyoiSUFJqcgA2FbbwMfvWsKmPQ1875JZXLdgYpe9jkNzw8PKTpsyomNbIBhic00DayvreHnZaqZOmUKaGWbgM8Nn4R5RM8NnRpqPjtv7Glr5x1vb+eK9K/jRE+u45pQJfOiU8YwsyOoxXucc3/73KlbuPMAfPjyfKSPzo35OREQSyQnjh3HC+GFcOW8cP396PS9t2MNPn1x/yD7vmlXMpceX4BzkZqYxsShvwD8fy6vr+cTfllKUn8mHT9N6pCKSHJSUisTYq+U13PT3ZTgHd33kZBZOLerT89PTfEwvLmB6cQFD9m9kUR/nZH709Em8tHEPf31tK//z7AZ++0I5F88dww0LJnFs6ZAun3P3G9u4f2kFnztnKufPKu7T64mIDGaZ6T6++e6ZfPPdMznQ2MbL5XvYVtvIqp0H+M+q3Ty9pqpj3zSf8ZGFE/nYGZMpHqAq30+srKQ1GOKJz52hyuIikjSUlIrEiHOOO1/byg8eX8vkojz+eN38iJZSiTafz1g0YxSLZoxi05567nptK/9aWsGDy3Yyf8Iwblg4iQtmF5PuDUl7a+tebnl0DWfPGMkXzps+4PGKiCSKIbkZXHxcuFr43oZWstJ9zBhdyPTicNX0nz61nj+8vIXn11Xz0E0LKRyAaQ6vltcwa0yhElIRSSpKSkVioDUQ4jsPr+Kfb+3gvJmj+J8PHJ8QczKnjMznlsvm8OULZvCvJRXc+dpWbvr7MsYMyeba0yZwzjGj+PQ9yygdlsMvrzphUFSgFBEZCMPzMvnlVSccsu3fU0bw8sYabvjrW3zyb0u5aM5o3jV7dMx6TZtag7y9fT/XL5wYk+OLiMSLklKRKNvjb+FTdy9lybZ9fObsqXzp/OkJl9wVZmfw0dMncf2CiTy/rpq/vraFnz65np8+uZ7czDTu/ugpDMmJfxItIpLIzIwzp4/k2++ZyU+fXM9rm2r5wWNruerkcXz4tAlMKsqPaiGiJdv20hoMHVJ3QEQkGSgpFYmiVTsPcONdS9jb2MqvP3gCl8wtiXdIPUrzGefPKub8WcWs3+3nn29t56zpI5kxuiDeoYmIDBo3LAz/yLe5poE/vryZu17fxl2vb2PqqHzee8JYLp1bwrjhuf0+vnOOVTvrePydStJ9xskTh0cxehGR+FNSKnKUnHPU1Lfy4oY9fPvfKxmem8n9n1zAnLFdFxFKVDNGF/DdS2bHOwwRkUHJzJgyMp//ft9xXHxcCWt21XH/0gp+9tR6lm3bx5+uP6nfx35ubTUfu2sJACdPHE5elr6+iUhy0aeaSAQONLVRsa+RHXubvL+N7NjXxI69jVTsa6KpLQjA/AnD+N0183pdckVERJLXwqlFLJxaxMfOmMQPH1/Ln17ZwgNLK3CEp3i0BIKUDM2hdFgOk4ryGJmfxdraIGvLNnHOMaM6RqtsrWng8ZWVPLOmipEFWfzgstmD7gdPEZFIJFVS+qV7l7O7qoUnalaQ5vORkWak+3ykpxnpPiM9zef9NTJ8PtJ85q3vGLuYnINAyBEIhry/jkAo1LGtzbsfDLnw7WAIs3BcXcWf4TPSvG3tj2/a1kb5y5uPOGbH63V67TbvtXyd1rlMM8PnrW2Z5q172R6Dz8LVWwNBR2sgFP4vGP7b0nE7eMj21kAIB+RmppOXmUZuVvhvTmYaeZnp5GZ5fzPTwvtkpZGVnkYg1Om4nV7r0PvBjttVVc08UrUcn+/wtTrx1vE8uG6nz3uTQ84RDIX/OucIOkfIhXs7g6Hw7ZBzhEKOprYgFV7iWdccOOR9LchKp3R4LpOK8jhj2kjGDc9h4og8Fk4tIjM9/guri4hI/JkZNyycyENv7+TL/1oRwTPW8ZMn13H1KeMZXZjNr18opzUQAuAzZ0/lwjljYhuwiEicxCwpNbMLgV8BacAfnXM/jtVrtVtesZ+6+iBbGmoOJmaH/HWxDiEiZnQkxeHk0kuWvcQTCCeXXSSzbcFu2rB27SF309qP157Mppm3LZzQuk7JV0ci5g7eDoZcxz7BkCPdZ2Sm+w7+l+YjMz2NzHQfWWk+cjPTGdqxPdyGxtYgTW0B6pra2H2giYaWII2tARpbg7R4F9lItR83s9NrNDeH2NG8F+fwEsrD2nJY25zDS7TDCXn7bZ+XyPrs4I8UPgu3d+zQHE4YP5Rxw3IZNzzX+5vDkJwMLJa/ZoiISFIoHZbLy187m3W7/YzIy2RUYRYZaT4q9zdTsa+R8j31HGhso6piK5++7HR+9exG7lm8veP51546gYVTizhv5qg4tkJEJLZikpSaWRrwW+B8oAJ4y8wecc6ticXrtXv+y4soKytj0aJFXT7e3hsWCDnaguEew+AAJKrtiWC499N31JX4gl78gZAjGHS89MornHXm6QcTXZ8lXLXXwwWCIRrbgjS2BGloDdDcFuwy8Wy/3VUC2NN7LSIikijystKZN2HYIdvGj8hl/IhcFkwtAqCsbCclQ3P4yRXH8Z1LZpHmM/b4W46qQJKIyGARq57Sk4Fy59xmADP7J3AZENOktDdm4Z7J9DTIzkiLZyhHJTzs+GD8BZk2IAt2R1N6mo/CNN+gi1tERCTW2gsZKSEVkVQRq6R0LLCj0/0K4JTOO5jZjcCNAMXFxZSVlUXlhevr66N2rMEiFdsMqdnuVGwzpGa7U7HNkLrtFhERSWWxSkq7Gjt6yDhZ59wdwB0A8+fPd9EahpmKQzpTsc2Qmu1OxTZDarY7FdsMqdtuERGRVBarMqEVwLhO90uBXTF6LRERERERERmkYpWUvgVMM7NJZpYJXAU8EqPXEhERERERkUEqJsN3nXMBM/sM8BThJWH+7JxbHYvXEhERERERkcErZuuUOueeAJ6I1fFFRERERERk8IvV8F0RERERERGRXikpFRERERERkbhRUioiIiIiIiJxo6RURERERERE4sacc/GOATPbA2yL0uGKgJooHWuwSMU2Q2q2OxXbDKnZ7lRsM0Sv3ROccyOjcJyUpWvzgNM5iozOU2R0nnqncxSZaJ6nbq/NCZGURpOZLXHOzY93HAMpFdsMqdnuVGwzpGa7U7HNkLrtTnZ6X3uncxQZnafI6Dz1TucoMgN1njR8V0REREREROJGSamIiIiIiIjETTImpXfEO4A4SMU2Q2q2OxXbDKnZ7lRsM6Ruu5Od3tfe6RxFRucpMjpPvdM5isyAnKekm1MqIiIiIiIig0cy9pSKiIiIiIjIIJE0SamZXWhm682s3My+Ee94BoqZbTWzlWa23MyWxDueWDCzP5tZtZmt6rRtuJk9Y2Ybvb/D4hljLHTT7u+Z2U7v/V5uZu+OZ4zRZmbjzOwFM1trZqvN7PPe9qR+v3tod9K+32aWbWZvmtkKr823eNuT+r1ONal6be5KX69lZnazd97Wm9kF8Yl6YPXnGpCi56nPn5+peJ4AzCzNzN42s8e8+zpHh+kql4jHeUqKpNTM0oDfAhcBs4APmtms+EY1oM52zh2fxGWt/wpceNi2bwDPOeemAc9595PNXzmy3QD/473fxzvnnhjgmGItAHzZOTcTOBW4yft/Odnf7+7aDcn7frcA5zjn5gLHAxea2akk/3udMnRtPsJfifBa5p2nq4DZ3nNu985nsuvTNSCFz1OfPj9T+DwBfB5Y2+m+zlHXDs8lBvw8JUVSCpwMlDvnNjvnWoF/ApfFOSaJEufcS8DewzZfBtzp3b4TuHwgYxoI3bQ7qTnnKp1zy7zbfsIXkrEk+fvdQ7uTlgur9+5meP85kvy9TjG6NnfSx2vZZcA/nXMtzrktQDnh85nU+nENSNXz1NfPz5Q8T2ZWCrwH+GOnzTpHkRnw85QsSelYYEen+xUk+Re6ThzwtJktNbMb4x3MACp2zlVC+CIGjIpzPAPpM2b2jjcULGmHNprZROAEYDEp9H4f1m5I4vfbG1a1HKgGnnHOpdR7nQJS+docqe7+vaf8uYvwGpCy56mPn5+pep5+CXwNCHXapnN0pK5yiQE/T8mSlFoX21KlrPBC59yJhIdH3WRmZ8Y7IImp3wFTCA/XqQR+EddoYsTM8oEHgC845+riHc9A6aLdSf1+O+eCzrnjgVLgZDObE+eQJLpS+dp8tFL63PXhGpCy56mPn58pd57M7GKg2jm3NNKndLEtqc9RJ33JJWJ2npIlKa0AxnW6XwrsilMsA8o5t8v7Ww08ROoMNagyszEA3t/qOMczIJxzVd6FKAT8gSR8v80sg/CXkXuccw96m5P+/e6q3anwfgM45/YDZYTnpyT9e51CUvba3Afd/XtP2XPXx2tAyp6ndhF+fqbieVoIXGpmWwlPHTjHzO5G5+gI3eQSA36ekiUpfQuYZmaTzCyT8ATcR+IcU8yZWZ6ZFbTfBt4FrOr5WUnjEeA67/Z1wMNxjGXAtH9AeN5Lkr3fZmbAn4C1zrnbOj2U1O93d+1O5vfbzEaa2VDvdg5wHrCOJH+vU0xKXpv7qLt/748AV5lZlplNAqYBb8YhvgHVj2tAqp6nvn5+ptx5cs7d7Jwrdc5NJPzZ87xz7hp0jg7RQy4x4OcpPRoHiTfnXMDMPgM8BaQBf3bOrY5zWAOhGHgo/BlOOvB359yT8Q0p+szsH8AioMjMKoDvAj8G7jOzjwLbgSvjF2FsdNPuRWZ2POGhEluBT8QrvhhZCFwLrPTmygB8k+R/v7tr9weT+P0eA9zpVe3zAfc55x4zs9dJ7vc6ZaTwtblLfbmWOedWm9l9wBrCFWlvcs4F4xL4wOrTNSCFz1OfPj9T+Dx1Rf+WDtVlLmFmbzHA58mcS5Xh0iIiIiIiIpJokmX4roiIiIiIiAxCSkpFREREREQkbpSUioiIiIiISNwoKRUREREREZG4UVIqIiIiIiIicaOkVEREREREROJGSamIiIiIiIjEjZJSERERERERiZv/Dxg7zG1d4i5rAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████████████████████████████████| 30001/30001 [04:43<00:00, 105.77it/s]\n"
     ]
    }
   ],
   "source": [
    "train_agent_noisy(env, agent, target_network, optimizer, td_loss_fn=td_loss_dqn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "final score(Noisy DQN): 111\n"
     ]
    }
   ],
   "source": [
    "final_score = evaluate(\n",
    "  make_env(ENV_NAME),\n",
    "  agent, n_games=30, greedy=True, t_max=1000\n",
    ")\n",
    "print('final score(Noisy DQN): {:.0f}'.format(final_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "In this notebook we saw how to train a DQN agent with experience replay and target networks. We also learnt a new way to explore i.e. to introduce some amount of randomness in the linear layer weights instead of using epsilon-greedy exploration. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
